golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(11.1): persist human verification items as UAT

Browse Source
This commit is contained in:
Jakub Zych
2026-10-01 09:36:25 +02:00
parent 571a2e56a0
commit 565ce982d9
2 changed files with 309 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-UAT.md Normal file
View File

@@ -0,0 +1,52 @@
---
status: testing
phase: 11.1-summercms-documentation-for-humans-and-ai-agents
source: [11.1-VERIFICATION.md]
started: 2026-10-01T07:36:12Z
updated: 2026-10-01T07:36:12Z
---
## Current Test
number: 1
name: Run summer docs:serve, open the printed address, search for a module name and a CLI command, and use the arrow keys and Enter.
expected: |
Rows show section, title, heading and a 2-line excerpt with mark highlights; arrow keys move the selection and Enter opens the page; the live region announces the count; empty and zero-result states show the UI-SPEC copy.
awaiting: user response
## Tests
### 1. Run summer docs:serve, open the printed address, search for a module name and a CLI command, and use the arrow keys and Enter.
expected: Rows show section, title, heading and a 2-line excerpt with mark highlights; arrow keys move the selection and Enter opens the page; the live region announces the count; empty and zero-result states show the UI-SPEC copy.
result: [pending]
### 2. Cycle the theme toggle through light, dark and system, reload in each mode, and check a guide page and an API page at 1280px, 1024px and 375px.
expected: No flash of the wrong theme; the choice persists; UI-SPEC contrast pairs hold in both themes.
result: [pending]
### 3. Disable JavaScript and load a guide page below 1024px.
expected: The sidebar renders above the content and the search and theme buttons are hidden.
result: [pending]
### 4. Read llms.txt, one page .md and llms-full.txt as an agent would.
expected: Spec shape with predictable .md URLs that are useful without HTML.
result: [pending]
### 5. Read Setup, Backend, Database and Services, and follow docs/setup/porting-a-plugin.md top to bottom.
expected: WinterCMS docs tone, accurate to the module READMEs, ending at the same acme.blog plugin as docs/examples/blog.
result: [pending]
### 6. Resolve the seven judgment-tier prohibitions in the report body.
expected: Each holds. The verifier's non-authoritative verdict is holds for all seven.
result: [pending]
## Summary
total: 6
passed: 0
issues: 0
pending: 6
skipped: 0
blocked: 0
## Gaps

422
.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VERIFICATION.md
View File

@@ -1,8 +1,8 @@
---
phase: 11.1-summercms-documentation-for-humans-and-ai-agents
verified: 2026-09-30T23:06:42Z
status: gaps_found
score: 10/13 must-haves verified
verified: 2026-10-01T07:31:53Z
status: human_needed
score: 13/13 must-haves verified
covered_files:
- ".gitignore"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-01-PLAN.md"
@@ -17,6 +17,12 @@ covered_files:
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-05-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-06-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-06-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-07-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-07-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-REVIEW-DISPOSITION.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-REVIEW.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md"
- ".planning/todos/pending/readme-go-fences-src.md"
- "CLAUDE.md"
- "README.md"
- "cmd/summer/docs.go"
@@ -116,6 +122,8 @@ covered_files:
- "internal/docsite/docsite_test.go"
- "internal/docsite/emit.go"
- "internal/docsite/emit_test.go"
- "internal/docsite/fences.go"
- "internal/docsite/fences_test.go"
- "internal/docsite/highlight.go"
- "internal/docsite/highlight_test.go"
- "internal/docsite/load.go"
@@ -126,6 +134,49 @@ covered_files:
- "internal/docsite/serve_test.go"
- "internal/docsite/snippet.go"
- "internal/docsite/snippet_test.go"
- "internal/docsite/testdata/violations/command-bin-no-dot/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-bin-no-dot/want.txt"
- "internal/docsite/testdata/violations/command-env-prefix/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-env-prefix/want.txt"
- "internal/docsite/testdata/violations/command-flag-first/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-flag-first/want.txt"
- "internal/docsite/testdata/violations/command-go-run/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-go-run/want.txt"
- "internal/docsite/testdata/violations/command-in-callout/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-in-callout/want.txt"
- "internal/docsite/testdata/violations/go-fence-callout-no-src/docs/extras/faq.md"
- "internal/docsite/testdata/violations/go-fence-callout-no-src/want.txt"
- "internal/docsite/testdata/violations/go-fence-golang/docs/extras/faq.md"
- "internal/docsite/testdata/violations/go-fence-golang/want.txt"
- "internal/docsite/testdata/violations/identifier-wrong-case/docs/extras/faq.md"
- "internal/docsite/testdata/violations/identifier-wrong-case/modules/demo/open.go"
- "internal/docsite/testdata/violations/identifier-wrong-case/want.txt"
- "internal/docsite/testdata/violations/snippet-blockquote/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-blockquote/want.txt"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/modules/demo/demo_test.go"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/modules/demo/ignored_test.go"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/want.txt"
- "internal/docsite/testdata/violations/snippet-build-ignored/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-build-ignored/modules/demo/broken.go"
- "internal/docsite/testdata/violations/snippet-build-ignored/want.txt"
- "internal/docsite/testdata/violations/snippet-callout-drift/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-callout-drift/want.txt"
- "internal/docsite/testdata/violations/snippet-callout-missing/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-callout-missing/want.txt"
- "internal/docsite/testdata/violations/snippet-example-no-output-helper/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-example-no-output-helper/modules/demo/never_test.go"
- "internal/docsite/testdata/violations/snippet-example-no-output-helper/want.txt"
- "internal/docsite/testdata/violations/snippet-example-no-output-region/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-example-no-output-region/modules/demo/never_test.go"
- "internal/docsite/testdata/violations/snippet-example-no-output-region/want.txt"
- "internal/docsite/testdata/violations/snippet-list-item/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-list-item/want.txt"
- "internal/docsite/testdata/violations/snippet-readme-src/modules/demo/README.md"
- "internal/docsite/testdata/violations/snippet-readme-src/want.txt"
- "internal/docsite/testdata/violations/snippet-testdata-go/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-testdata-go/modules/demo/testdata/fixture_test.go"
- "internal/docsite/testdata/violations/snippet-testdata-go/want.txt"
- "internal/docsite/theme/assets/search.js"
- "internal/docsite/theme/assets/site.css"
- "internal/docsite/theme/assets/site.js"
@@ -179,228 +230,267 @@ covered_files:
- "modules/wire/example_test.go"
- "modules/wristband/example_test.go"
- "scripts/check-phase11.1.sh"
covered_digest: "v2:sha256:c68da6c413348d7d4d0d272df3d007aa82e2f9ca5aa9f9ab79eefc797d2cb7d0"
covered_digest: "v2:sha256:69e8741c6e6d238b3ae5164f594e9dce56016ddf6cda5b738f3fb81f3fec780e"
behavior_unverified: 0
overrides_applied: 0
gaps:
- truth: "DOCS-04 / D-07 / D-18 policy: a test fails on a missing or drifted src= snippet, and every Go fence in a docs/ page must carry src= (plan 01 truth 5, plan 02 truth 5)"
status: failed
reason: "Fence discovery in the checkers is a line scanner that only strips leading spaces. It never sees a fence inside a blockquote or callout (`> ```go src=...`), and the go-fence policy compares the language word to the literal `go`, so a ```golang fence is exempt. goldmark still renders both, and fenceAnnotator still adds the source caption. Reproduced on a git-archive copy of HEAD: a `> [!TIP]` callout holding `go src=modules/bonfire/example_test.go#ExampleCall` with body BOGUS passed `summer docs:build --check`, and `docs:build` wrote setup/installation.html with BOGUS under a figcaption linking modules/bonfire/example_test.go. A ```golang fence with no src= also passed. Module README src= fences are annotated but never checked (snippet.go skips README pages). The acceptance test's independent scanner (docsGoFences/pageFences) has the same blind spot, so SC4 cannot catch it. No real page uses either form today, so the content is correct, but the guarantee DOCS-04 states is not enforced. (Review CR-01, WR-04.)"
artifacts:
- path: "internal/docsite/render.go"
issue: "scanFences/openFence skip blockquote-prefixed fences (and accept 4-space-indented backticks, IN-02); fenceAnnotator captions every src= fence regardless of whether it was checked"
- path: "internal/docsite/snippet.go"
issue: "checkFences relies on scanFences; checkSnippets skips module README pages while their src= fences are still captioned"
- path: "internal/docsite/check_policy.go"
issue: "go-fence-needs-src policy matches only fields[0] == \"go\"; golang (chroma alias) bypasses it"
- path: "cmd/summer/phase11_1_acceptance_test.go"
issue: "pageFences does not strip `>` prefixes and docsGoFences checks lang == \"go\" only"
missing:
- "Collect fences from the goldmark AST (*ast.FencedCodeBlock) for snippet, policy and command checks, or refuse src= and Go fences that are not top-level blocks of a docs page"
- "Refuse (or check) src= fences in module READMEs, or stop fenceAnnotator captioning them"
- "Normalise the fence language through chroma so any Go-lexer alias needs src="
- "Planted fixtures in internal/docsite/testdata/violations for a callout src= fence with a drifted body, a callout src= fence naming a missing file, a callout go fence without src=, and a golang fence; update the acceptance scanner to strip `>`"
- truth: "D-07 proof of execution: a src= target must be compiled source, and a region or helper that no Test or Example function runs is a snippet problem (plan 01 truth 12; SC4 'compiled and run by go test')"
status: partial
reason: "isRoot counts every Example* function as a root, but go test never runs an Example without an // Output: comment. Extract checks only that a non-test .go file sits in the root module next to a _test.go; it ignores //go:build constraints and GOOS/GOARCH suffixes. Reproduced on a copy of HEAD: Example_neverRun (no Output) calling neverRunHelper made src=...#neverRunHelper pass `--check`; modules/bonfire/broken.go with `//go:build ignore` and a call to an undefined function passed as src=...#Broken. No real target is affected today: an AST scan of every example*_test.go and docs/examples test file found no Example without Output, and no src= target has a build constraint. (Review WR-01, WR-02.)"
artifacts:
- path: "internal/docsite/snippet.go"
issue: "isRoot treats Examples without output as run; Extract does not consult go/build for build constraints"
missing:
- "Build the reachability roots from Test* functions plus Examples that doc.Examples reports with Output or EmptyOutput"
- "Confirm a non-test src= file is in the default build via go/build ImportDir (GoFiles/TestGoFiles/XTestGoFiles)"
- "Planted fixtures for both cases"
- truth: "DOCS-05 / plan 02 truths 1 and 3: an inline pkg.Ident whose Ident does not exist fails, and a `summer <cmd>` token in an sh fence must be a toolCommands() name"
status: partial
reason: "The identifier checker falls back to `go doc ./<dir> <query>` without -c, and go doc matches a lowercase query letter case-insensitively. Reproduced on a copy of HEAD: `lagoon.Openfromapp` passed `--check`, although the identifier does not exist in modules/lagoon (`go doc -c` exits 1). The command checker matches `summer <name>` only at the start of a command with no flag first: `FOO=1 summer no:such` and `summer --root . no:such2` in an sh fence both passed. Links, anchors and forbidden names fail correctly (mutation-tested). No real page exploits these holes: all 704 module-package spans in docs/, module READMEs and the root README resolve with `go doc -c`, and no sh fence uses an env-var prefix, a flag before the command, `go run ./cmd/summer` or `bin/` without `./`. (Review WR-03, WR-05.)"
artifacts:
- path: "internal/docsite/check_identifiers.go"
issue: "goDoc runs `go doc` without -c (line 301)"
- path: "internal/docsite/check_commands.go"
issue: "commandToken skips VAR=value prefixes, flags before the command name, `go run ./cmd/summer` and `bin/<app>`"
missing:
- "Pass -c to the go doc fallback"
- "Strip leading VAR=value assignments and a `go run ./cmd/summer` prefix, skip flags before the command word, accept bin/<app>"
- "Planted fixtures for a wrong-case identifier and each command form"
re_verification:
previous_status: gaps_found
previous_score: 10/13
gaps_closed:
- "DOCS-04 fence discovery: AST collection refuses nested and README src= fences, Go-lexer aliases need src=, and captions cover only checked fences"
- "D-07 proof of execution: src= targets must be in the default build and reached from a Test or an Example with output"
- "DOCS-05 name forms: go doc -c is case-sensitive, and command words cover env prefixes, flags, go run ./cmd/summer and bin/{app}"
gaps_remaining: []
regressions: []
decision_coverage:
honored: 18
total: 18
not_honored: []
advisory:
- finding: "The identifier index still reads build-ignored Go files, so a declaration go test never compiles can still satisfy an identifier span."
category: other
reason: "Recorded by plan 11.1-07 as outside the gap-closure contract. It is not one of the three prior gaps. No red test was run for it on this pass, so it does not reopen DOCS-05."
evidence_status: "none provided"
human_verification:
- test: "Run `summer docs:serve`, open the printed address, press the search trigger, search for a module name (e.g. lagoon) and a CLI command (e.g. migrate:status); use the arrow keys and Enter."
expected: "Rows show section › title, heading and a 2-line excerpt with <mark> highlights; arrow keys move the selection and Enter opens the linked page; the live region announces the result count; empty and zero-result states show the UI-SPEC copy."
why_human: "search.js has no JS test runner (no Node toolchain by design); Go tests only assert markup and asset presence."
- test: "In docs:serve, cycle the theme toggle through light, dark and system; reload in each mode; check a guide page and an API reference page at 1280px, 1024px and 375px."
expected: "No flash of the wrong theme on reload (theme-init.js applies the mode before first paint); the choice persists; UI-SPEC contrast pairs hold in both themes."
why_human: "Visual and timing behaviour; not observable by grep or Go tests."
- test: "Run summer docs:serve, open the printed address, search for a module name and a CLI command, and use the arrow keys and Enter."
expected: "Rows show section, title, heading and a 2-line excerpt with mark highlights; arrow keys move the selection and Enter opens the page; the live region announces the count; empty and zero-result states show the UI-SPEC copy."
why_human: "Plan 02 and plan 06 backstop truths, and the two manual rows in 11.1-VALIDATION.md. search.js has no JS test runner."
- test: "Cycle the theme toggle through light, dark and system, reload in each mode, and check a guide page and an API page at 1280px, 1024px and 375px."
expected: "No flash of the wrong theme; the choice persists; UI-SPEC contrast pairs hold in both themes."
why_human: "Plan 02 backstop truths. Visual timing is not observable from Go tests."
- test: "Disable JavaScript and load a guide page below 1024px."
expected: "The sidebar renders above the content; the search and theme buttons are hidden."
why_human: "Visual no-JS layout (plan 02 backstop truth)."
- test: "Read llms.txt, one page's .md and llms-full.txt as an AI agent would (raw fetch)."
expected: "Spec shape (H1, blockquote, H2 link lists) with predictable .md URLs that are useful without the HTML."
why_human: "Plan 01 backstop truth (usefulness is non-inferable from structure)."
- test: "Read the Setup, Backend, Database and Services pages and follow docs/setup/porting-a-plugin.md top to bottom as a WinterCMS developer."
expected: "WinterCMS docs tone; accurate to the module READMEs; the walkthrough ends with the same acme.blog plugin as docs/examples/blog."
why_human: "Plan 03/04/05 backstop truths (editorial quality and followability)."
- test: "Review the six judgment-tier prohibitions listed under Prohibitions in the report body."
expected: "Each holds (the verifier's non-authoritative LLM-judge verdict is 'holds' for all six)."
why_human: "Judgment-tier prohibitions need explicit human resolution; the verifier's verdict is not authoritative."
expected: "The sidebar renders above the content and the search and theme buttons are hidden."
why_human: "Plan 02 backstop truth (no-JS layout)."
- test: "Read llms.txt, one page .md and llms-full.txt as an agent would."
expected: "Spec shape with predictable .md URLs that are useful without the HTML."
why_human: "Plan 01 backstop truth. Usefulness is non-inferable from structure. unverified — held-out test recommended."
- test: "Read Setup, Backend, Database and Services, and follow docs/setup/porting-a-plugin.md top to bottom."
expected: "WinterCMS docs tone, accurate to the module READMEs, ending at the same acme.blog plugin as docs/examples/blog."
why_human: "Plans 03, 04 and 05 backstop truths."
- test: "Resolve the seven judgment-tier prohibitions in the report body."
expected: "Each holds. The verifier's non-authoritative verdict is holds for all seven."
why_human: "unverified-prohibition — human review recommended. Judgment-tier prohibitions are not a silent pass."
---
# Phase 11.1: SummerCMS documentation for humans and AI agents Verification Report
**Phase Goal:** SummerCMS has a WinterCMS-style documentation set that serves both humans and AI agents. The Markdown source lives in `summercms.go/docs/` and a `summer` CLI command builds it into a static site with sidebar navigation, search, `llms.txt`/`llms-full.txt` and a raw `.md` per page. Every code example compiles and is tested, and a "Coming from WinterCMS" map plus an `acme/blog` porting walkthrough cover the migration path. It documents the framework as it stands after Phase 11 and never names a consuming application.
**Verified:** 2026-09-30T23:06:42Z
**Status:** gaps_found
**Re-verification:** No, initial verification
**Verified:** 2026-10-01T07:31:53Z
**Status:** human_needed
**Re-verification:** Yes — after gap closure (plan 11.1-07)
## Verdict in one paragraph
The documentation set exists and is correct today. There are 50 guide pages in 8 Winter-mirroring sections and 22 API reference pages ingested from the module READMEs. The build writes HTML, a `.md` per page, `llms.txt` and `llms-full.txt`, with no Node toolchain. Every one of the 106 Go fences in docs/ is a `src=` copy that is byte-checked against source that `go test ./...` compiles and runs. The concept map and the Docker-tested acme/blog walkthrough are in place, and no consuming application is named anywhere. The gaps are in the enforcement that DOCS-04 and DOCS-05 promise: "a test fails on a missing or drifted snippet" and "checkers fail on stale identifiers and unknown commands". I reproduced every review finding the orchestrator asked about on a `git archive HEAD` copy of the real tree, and each checker fails open. The worst case is CR-01: a `src=` fence inside a callout is published with a "verified source" caption over code nobody checked. No current page exploits any of these holes, so these gaps concern guard-rail correctness, not content correctness. Each fix is small and local to `internal/docsite`.
The three gaps from 2026-09-30 are closed in the code, and the phase gate is green. `collectFences` walks the goldmark AST the renderer uses. A `src=` fence inside a callout, blockquote or list item, and any `src=` fence in a module README, fails Check and is not captioned. Go-lexer aliases need `src=`. A `.go` target must be in the default build and reached from a `Test` or an Example with output. `go doc -c` is case-sensitive, and `commandWord` accepts env prefixes, cobra-style flags, `go run ./cmd/summer` and `bin/{app}`. `bash scripts/check-phase11.1.sh --all` printed `phase11.1 all passed` (exit 0), including a scratch-tree self-test that plants the old holes. Automated must-haves are 13/13. Status stays `human_needed` because the plans' backstop truths (search, theme, no-JS layout, editorial usefulness) and seven judgment-tier prohibitions still need a person. That is not a code gap.
## Goal Achievement
### Observable Truths
The scored set is the same 13 truths as the previous report. Plan 11.1-07's ten truths are the re-test of truths 6–8; they are listed under that table and are not a second denominator.
| # | Truth | Status | Evidence |
|----|-------|--------|----------|
| 1 | SC1: docs/ holds Markdown pages with frontmatter in Setup, Architecture, Plugins, Backend, Database, Services, Console, API reference, and every framework module is reachable from the sidebar | ✓ VERIFIED | `docs/site.yaml` lists the 8 sections in that order; 50 guide pages; built site has `api/<m>.html` for all 22 top-level modules. `TestPhase11_1Acceptance/SC1` PASS (run by me, `-count=1`). Sub-packages (attach, typesense, centrifugo) are documented inside their parent README, not as separate pages (see Info). |
| 2 | SC2: a `summer` command builds a self-contained static site with sidebar, TOC, prev/next, client-side search and dark mode, no Node toolchain; only new dep is approved | ✓ VERIFIED (static); interaction to human UAT | `summer docs:build --out <scratch>` wrote 72 pages; theme markers asserted by `SC2` PASS; no `exec` of node/npm/npx (SC2 AST scan); `git log` of 11.1 commits shows go.mod gained only `alecthomas/chroma/v2` + indirect `dlclark/regexp2/v2` (goldmark predates the phase, from 04-03). Built HTML/CSS/JS loads nothing from a third-party origin (fonts vendored under /assets/fonts). Search interaction and theme flash are routed to human verification. |
| 3 | SC3/DOCS-03: `llms.txt`, `llms-full.txt` and a clean `.md` per page, with a test asserting sync with the page tree | ✓ VERIFIED | Built output inspected: `llms.txt` has `# SummerCMS`, a blockquote, H2 link lists; `setup/installation.md` starts `# Installation` / `> description`, no frontmatter. `TestDocsAIOutputsInSync` and `SC3` PASS. |
| 4 | SC4 (current content): every Go example in a docs/ page is compiled and run by `go test ./...` | ✓ VERIFIED | 106 ```go fences, 0 without `src=`, none inside blockquotes, no ```golang fences. All src= targets are in root-module packages (SC4 go-list check). AST scan (go/doc): no Example in any `example*_test.go` or docs/examples test lacks `// Output:`. No src= target has a build constraint. `go test -count=1 -run '^(Example|TestDocs)' ./modules/...` all ok, 0 SKIP. |
| 5 | SC5 / DOCS-07: the concept map and the acme/blog walkthrough exist, and the walkthrough's code is verified under SC4 | ✓ VERIFIED | `docs/setup/porting-a-plugin.md` (829 lines) covers plugin registration, model, migrations (plus a rollback column), routes, admin controller and console command; its fences are src= copies from `docs/examples/blog` (in-root package, no go.mod). `go test -count=1 -v ./docs/examples/blog/...` ran TestMigrateUpAndRollback, TestPostsRouteAgainstDatabase, TestPublishCommandAgainstDatabase and TestScaffoldLayout on Docker, all PASS, none skipped. `SC5` PASS. |
| 6 | DOCS-04 enforcement: a test fails on a missing or drifted snippet; every Go fence must carry src= | ✗ FAILED | Top-level fences are enforced: drift in the page and drift in the source both refuse, and `docs:sync` repairs them (mutation-tested). Fences inside callouts and blockquotes are never checked but are still captioned as verified source (CR-01, reproduced on real tree copy). ```golang bypasses the src= policy (WR-04, reproduced). README src= fences are captioned without being checked. |
| 7 | D-07 proof of execution: src= must name compiled source, and code no Test or Example runs is a snippet problem | ✗ FAILED (partial) | An Example without Output counts as a root (WR-01, reproduced). `//go:build ignore` files pass as compiled (WR-02, reproduced). Not exploited on the real tree. |
| 8 | DOCS-05: checkers fail on stale identifiers, broken links/anchors, unknown command names and forbidden names | ✗ FAILED (partial) | Links, anchors, forbidden names, unknown identifiers and plain `summer x`/`./bin/app x` forms all refuse (mutation-tested). Wrong-case identifiers pass via the `go doc` fallback without `-c` (WR-03: `lagoon.Openfromapp` passed). `FOO=1 summer no:such` and `summer --root . no:such2` pass (WR-05). Not exploited: all 704 module spans resolve with `go doc -c`. |
| 9 | DOCS-06: "Coming from WinterCMS" maps Winter concepts to SummerCMS, with checker-verified identifiers and explicit Not provided rows | ✓ VERIFIED | Concept table covers Plugin.php, $require, permissions/navigation, version.yaml, Eloquent, fields/columns.yaml, backend controllers, routes.php, middleware, config, lang, events, App::make, artisan, queues, scheduler, mail, settings, broadcasting, Scout, the HTTP client. A Not provided table covers CMS pages/themes, components, AJAX/Snowboard, media manager, import/export, reorder, collections, behaviours, cache and session, linking frontend-and-ajax.md. All `pkg.Ident` spans resolve case-sensitively. |
| 10 | DOCS-08: CLAUDE.md records the docs update rule and names the checkers | ✓ VERIFIED | CLAUDE.md `## Documentation` (HEAD, clean worktree) has the rule for "affected pages under `docs/`", names `go test ./cmd/summer -run TestDocsTree` and `summer docs:build --check`, and notes that config-key checking is deferred. The gate's `--claude` stage passes. |
| 11 | D-11: no page, snippet or built output names a consuming application | ✓ VERIFIED | `grep -rliE` over docs/, all module READMEs, the root README and the full built site finds nothing, except the checker's own pattern in `check_forbidden.go`. A planted name is refused (mutation-tested). |
| 12 | Phase-end tests: docsite coverage ≥85%, a planted fixture per rule, and acceptance subtests SC1–SC5 | ✓ VERIFIED | `go test -cover ./internal/docsite` gives 94.8%. `TestPlantedViolations` has 68 passing subtests. `TestPhase11_1Acceptance` SC1–SC5 PASS. |
| 13 | The gate: `check-phase11.1.sh --all` is green and `--named` refuses skipped or zero-match runs | ✓ VERIFIED | I ran `bash scripts/check-phase11.1.sh --all`. Every stage passed (preconditions, deps, self-test, docs, forbidden, claude, named, go) and it printed `phase11.1 all passed`, exit 0. `go_json_named` refuses skips and "no tests to run" (script lines 172–211). |
| --- | --- | --- | --- |
| 1 | SC1: docs/ holds Markdown pages with frontmatter in Setup, Architecture, Plugins, Backend, Database, Services, Console, API reference, and every framework module is reachable from the sidebar | ✓ VERIFIED | Regression: `docs/site.yaml` still lists those eight sections. `scripts/check-phase11.1.sh --all` ran `TestPhase11_1Acceptance` under `--named` (`-count=1`, skips refused) and printed `phase11.1 named passed`. `docs:build` wrote 72 pages. |
| 2 | SC2: a `summer` command builds a self-contained static site with sidebar, TOC, prev/next, client-side search and dark mode, no Node toolchain; only new dep is approved | ✓ VERIFIED | `--deps` passed (chroma/v2 only). `--docs` built the site. Search keystrokes and theme flash stay in human verification; markup is covered by the acceptance test the gate ran. |
| 3 | SC3/DOCS-03: `llms.txt`, `llms-full.txt` and a clean `.md` per page, with a test asserting sync with the page tree | ✓ VERIFIED | `--docs` requires `llms.txt`, `llms-full.txt` and `index.md`. `TestDocsAIOutputsInSync` is in the named set that passed. Usefulness of the raw files stays a backstop. |
| 4 | SC4 (current content): every Go example in a docs/ page is compiled and run by `go test ./...` | ✓ VERIFIED | `TestPhase11_1Acceptance` passed via `--named`. The acceptance scanner now marks blockquoted fences nested (`scanDocFences` / `TestAcceptanceFenceScanner`) and the real tree has no nested `src=` fence. |
| 5 | SC5 / DOCS-07: the concept map and the acme/blog walkthrough exist, and the walkthrough's code is verified under SC4 | ✓ VERIFIED | Both pages are still present. `--named` ran the blog package tests, including the Docker migrate/route/command tests, and refuses a skip. `phase11.1 named passed`. |
| 6 | DOCS-04 enforcement: a test fails on a missing or drifted snippet; every Go fence must carry src= | ✓ VERIFIED | Closed by plan 07 truths G1–G5 and G10. Self-test on a scratch copy of the real tree refused a callout `src=` fence (`must be a top-level block`) and a `golang` fence with no `src=`. `TestPlantedViolations` walks all 81 violation directories, including the new plants, and the self-test stage passed. |
| 7 | D-07 proof of execution: src= must name compiled source, and code no Test or Example runs is a snippet problem | ✓ VERIFIED | Closed by G6–G7. `loadTestGraph` roots are `isTestName` plus `doc.Examples` with `Output` or `EmptyOutput`. `checkBuiltGo` uses `build.ImportDir` on `build.Default` and refuses `testdata` and `_` directories. `TestSnippetRootsAndBuild` and the planted fixtures `snippet-example-no-output-helper`, `snippet-build-ignored` and `snippet-testdata-go` are in the corpus the self-test ran. |
| 8 | DOCS-05: checkers fail on stale identifiers, broken links/anchors, unknown command names and forbidden names | ✓ VERIFIED | Closed by G8–G9 for the two holes. `goDoc` runs `go doc -c`. Self-test refused `lagoon.Openfromapp`, `FOO=1 summer no:such` and `summer --root . no:such`. Links, anchors and forbidden names were already refused and the same self-test still plants them. |
| 9 | DOCS-06: "Coming from WinterCMS" maps Winter concepts to SummerCMS, with checker-verified identifiers and explicit Not provided rows | ✓ VERIFIED | Regression: the Not provided table in `docs/setup/coming-from-wintercms.md` still links `docs/services/frontend-and-ajax.md`. Identifiers are inside the checkers the gate ran. |
| 10 | DOCS-08: CLAUDE.md records the docs update rule and names the checkers | ✓ VERIFIED | `CLAUDE.md` `## Documentation` still names `TestDocsTree` and `summer docs:build --check`. `--claude` passed. |
| 11 | D-11: no page, snippet or built output names a consuming application | ✓ VERIFIED | `grep -rliE` over `docs/**/*.md` found no consuming-application spelling. `--forbidden` passed. A planted name is still in the self-test. |
| 12 | Phase-end tests: docsite coverage ≥85%, a planted fixture per rule, and acceptance subtests SC1–SC5 | ✓ VERIFIED | This pass: `go test -count=1 -cover ./internal/docsite` reported `coverage: 94.1% of statements`. 81 violation directories. `TestPlantedViolations` and `TestPhase11_1Acceptance` passed inside the gate. |
| 13 | The gate: `check-phase11.1.sh --all` is green and `--named` refuses skipped or zero-match runs | ✓ VERIFIED | This pass ran `bash scripts/check-phase11.1.sh --all`. Every stage passed and it printed `phase11.1 all passed`, exit 0. `detect_json` still exits on skip, "no tests to run" and a missing required name. |
**Score:** 10/13 truths verified (0 present, behavior-unverified).
**Score:** 13/13 truths verified (0 present, behavior-unverified)
Five backstop truths from the plans, which cover editorial usefulness, followability of the walkthrough, search rows and keys, no theme flash, contrast and the no-JS layout, are non-inferable. They are routed to human verification and not scored.
Five backstop truths (search rows and keys, no theme flash, contrast, no-JS layout, llms usefulness, editorial and walkthrough followability) stay non-inferable. They are not in the 13. They route to human verification with reason `insufficient_spec`.
### Plan 11.1-07 closure truths
All ten are verified. They are the detailed contract for truths 6–8, not extra score rows.
| # | Truth | Status | Evidence |
| --- | --- | --- | --- |
| G1 | Snippet, policy and command checks find every goldmark `*ast.FencedCodeBlock` | ✓ VERIFIED | `collectFences` in `internal/docsite/fences.go` walks the document from `newMarkdown`. `checkSnippets`, `checkPolicy` and `checkCommands` call it. `TestCollectFences` expects the callout, blockquote and list fences and rejects a four-space indented fence. |
| G2 | A non-top-level `src=` fence fails Check, docs:build --check and docs:sync, and sync writes nothing | ✓ VERIFIED | `nestedSrcMessage` in `checkFences` returns before `Extract`. `TestSyncParsedFences` expects one problem and an unchanged tree. Self-test plant on the real-tree copy was refused. |
| G3 | A module README `src=` fence is refused and no unchecked fence gets a figcaption | ✓ VERIFIED | `readmeSrcMessage` when `page.Module != ""`. `fenceAnnotator` sets `data-src` only when `Module == ""` and the parent is the document. `TestFenceCaptionsOnlyVerified` expects one guide caption and none on the API page. |
| G4 | Any fence whose chroma lexer is Go needs `src=`, at any depth; go-html-template, text and empty do not | ✓ VERIFIED | `goLang` uses `lexerFor`. `checkPolicy` applies it to every collected fence of a docs page. `TestGoLang` and `TestNestedFenceChecks` (callout `go`, `GO`, `main.go`). Self-test refused `golang`. |
| G5 | Shell fences inside callouts are command-checked; four or more leading spaces are not a fence | ✓ VERIFIED | `checkCommands` reads `f.code` from the AST for `sh`/`shell`/`bash`/`console`. `openFence` rejects indent above 3. `TestCollectFences` wants 6 fences, not 7. Plant `command-in-callout`. |
| G6 | Reachability roots are Test functions plus Examples with output, from test files in the default build | ✓ VERIFIED | `loadTestGraph` skips Examples with empty `Output` and no `EmptyOutput`, and only parses `TestGoFiles`/`XTestGoFiles`. `TestSnippetRootsAndBuild` asserts `ExampleNone` and `onlyFromIgnored` are not reachable. |
| G7 | A `.go` src= target fails when it is under `testdata` or `_`, or `build.ImportDir` does not list it | ✓ VERIFIED | `checkBuiltGo`. Plants `snippet-build-ignored`, `snippet-build-ignored-test-root`, `snippet-testdata-go`. |
| G8 | `go doc -c`, so `demo.Openfromapp` fails | ✓ VERIFIED | `check_identifiers.go` argv is `go`, `doc`, `-c`. `TestIdentifierGoDocCaseSensitive` expects `Openfromapp` false. Self-test refused `lagoon.Openfromapp`. Plant `identifier-wrong-case`. |
| G9 | Command word after `VAR=value` and flags, for `summer`, `go run ./cmd/summer`, `./bin/{app}` and `bin/{app}`; `summer --help` alone is not a command | ✓ VERIFIED | `commandWord` in `check_commands.go`. `TestCommandWord` covers those forms. Self-test refused the env-prefix and flag-first plants. |
| G10 | Each hole has a planted fixture; the real tree passes TestDocsTree, docs:build --check and TestPhase11_1Acceptance; the gate prints `phase11.1 all passed` | ✓ VERIFIED | 18 new plants are in `testdata/violations` and `TestPlantedViolations` runs every directory. Gate exit 0, quoted above. `scanDocFences` strips `>` and `goFenceLang` uses chroma. SC4 cross-checks figcaption counts. |
### Required Artifacts
| Artifact | Expected | Status | Details |
|----------|----------|--------|---------|
| `internal/docsite/docsite.go` | Build, Check, Sync entry points | ✓ VERIFIED | Called from cmd/summer/docs.go |
| `internal/docsite/load.go` | strict frontmatter, README ingestion | ✓ VERIFIED | `loadModules` discovers modules dynamically |
| `internal/docsite/render.go` | goldmark with shared slug IDs | ✓ VERIFIED (with gap) | `parser.WithIDs`; `scanFences` is the root of gap 1 |
| `internal/docsite/emit.go` | html, .md, llms*, search index | ✓ VERIFIED | outputs inspected |
| `internal/docsite/snippet.go` | src= parse, confine, extract, drift | ✓ VERIFIED (with gaps) | drift works top-level; gaps 1 and 2 |
| `internal/docsite/check_identifiers.go` | identifier index with go doc fallback | ✓ VERIFIED (with gap) | gap 3 (case) |
| `internal/docsite/check_links.go` | links and anchors | ✓ VERIFIED | mutation-tested |
| `internal/docsite/check_commands.go` | command names from toolCommands and runtime set | ✓ VERIFIED (with gap) | gap 3 (forms) |
| `internal/docsite/check_forbidden.go` | forbidden names over sources and outputs | ✓ VERIFIED | mutation-tested |
| `internal/docsite/highlight.go` | chroma/v2 NodeRenderer | ✓ VERIFIED | tok-* classes in built HTML |
| `internal/docsite/serve.go` | loopback server, 404, rebuild | ✓ VERIFIED | serve tests pass; flag `--allow-remote` registered |
| `cmd/summer/docs.go` | docs:build, docs:sync, docs:serve | ✓ VERIFIED | in `toolCommands()`; binary built and run |
| `docs/setup/coming-from-wintercms.md` | concept map | ✓ VERIFIED | |
| `docs/setup/porting-a-plugin.md` | walkthrough | ✓ VERIFIED | |
| `docs/examples/blog/*` | in-root verified plugin | ✓ VERIFIED | Docker tests ran |
| `internal/docsite/violations_test.go` | TestPlantedViolations | ✓ VERIFIED | has no fixtures for gaps 1–3 |
| `cmd/summer/phase11_1_acceptance_test.go` | SC1–SC5 | ✓ VERIFIED (with gap) | scanner shares the blind spot in gap 1 |
| `scripts/check-phase11.1.sh` | --self-test, --named, --all | ✓ VERIFIED | ran --all |
| --- | --- | --- | --- |
| `internal/docsite/fences.go` | AST fence discovery | ✓ VERIFIED | `func collectFences`; wired from snippet, policy and command checks |
| `internal/docsite/snippet.go` | top-level src=, README refusal, build membership, Example roots | ✓ VERIFIED | `build.ImportDir`, `doc.Examples`, `nestedSrcMessage`, `readmeSrcMessage` |
| `internal/docsite/highlight.go` | `goLang` via the highlighter lookup | ✓ VERIFIED | `func goLang`; `checkPolicy` calls `goLang(` |
| `internal/docsite/check_commands.go` | `commandWord` | ✓ VERIFIED | env prefix, flags, `go run ./cmd/summer`, `bin/` |
| `internal/docsite/check_identifiers.go` | case-sensitive go doc fallback | ✓ VERIFIED | `"doc", "-c"` |
| `internal/docsite/render.go` | captions only on verified fences | ✓ VERIFIED | `Module == ""` and document parent |
| `internal/docsite/testdata/violations/snippet-callout-drift/want.txt` | CR-01 plant | ✓ VERIFIED | message contains `top-level block` |
| `cmd/summer/phase11_1_acceptance_test.go` | independent scanner and caption cross-check | ✓ VERIFIED | `TestAcceptanceFenceScanner`, `scanDocFences` |
| `scripts/check-phase11.1.sh` | gate plants the holes; `--all` | ✓ VERIFIED | ran this pass |
| `internal/docsite/docsite.go` | Check calls `checkSnippets` after parse | ✓ VERIFIED | call at `checkContent`; not left only in assemble |
| `docs/setup/coming-from-wintercms.md` | concept map | ✓ VERIFIED | regression, file present, Not provided rows |
| `docs/setup/porting-a-plugin.md` | walkthrough | ✓ VERIFIED | regression, file present |
| `docs/examples/blog/*` | in-root plugin | ✓ VERIFIED | named blog tests passed inside the gate |
| `CLAUDE.md` | documentation rule | ✓ VERIFIED | `--claude` passed |
### Key Link Verification
| From | To | Via | Status |
|------|----|-----|--------|
| cmd/summer/docs.go | internal/docsite | docsite.Build / Sync / Check | WIRED |
| cmd/summer/docs.go | check_commands.go | docsCommands() → toolCommands() + module command constructors | WIRED |
| render.go | goldmark | parser.WithIDs(slug IDs), shared with check_links | WIRED |
| highlight.go | chroma/v2 | lexers + tokenise → tok-* | WIRED |
| docs/setup/installation.md | modules/bonfire/example_test.go | src=…#ExampleCall | WIRED (drift is detected) |
| docs/setup/porting-a-plugin.md | docs/examples/blog/* | 15+ src= fences | WIRED |
| docs/examples/blog/scaffold_layout_test.go | internal/build | build.Make* | WIRED (TestScaffoldLayout PASS) |
| docs/setup/coming-from-wintercms.md | docs/services/frontend-and-ajax.md | Not provided rows link it | WIRED |
| scripts/check-phase11.1.sh | TestPlantedViolations | --self-test / --named | WIRED |
| From | To | Via | Status | Details |
| --- | --- | --- | --- | --- |
| `internal/docsite/render.go` | `internal/docsite/snippet.go` | caption set equals the fences `checkSnippets` compares | WIRED | `Module == ""` and `KindDocument` in `fenceAnnotator`; `TestFenceCaptionsOnlyVerified` |
| `internal/docsite/check_policy.go` | `internal/docsite/highlight.go` | `goLang` reuses `lexerFor` | WIRED | `goLang(` at the policy loop |
| `internal/docsite/snippet.go` | `go/build` and `go/doc` | `build.ImportDir` and `doc.Examples` | WIRED | `buildPackage`, `exampleHasOutput`, `loadTestGraph` |
| `internal/docsite/check_identifiers.go` | `go doc -c` | `goDoc` argv | WIRED | `exec.CommandContext(ctx, "go", "doc", "-c", ...)` |
| `scripts/check-phase11.1.sh` | new tests and fixtures | `--named` list includes `TestNestedFenceChecks`, `TestCommandWord`, `TestSnippetRootsAndBuild` | WIRED | `--named` passed |
| `cmd/summer/docs.go` | `internal/docsite` | `Build` / `Check` / `Sync` | WIRED | regression; gate builds `./cmd/summer` and runs `docs:build --check` |
| `docs/setup/porting-a-plugin.md` | `docs/examples/blog` | `src=` fences | WIRED | acceptance SC4/SC5 inside the gate |
### Data-Flow Trace (Level 4)
The docsite does not render application data from a database. Page HTML is produced from the Markdown sources.
| Artifact | Data Variable | Source | Produces Real Data | Status |
| --- | --- | --- | --- | --- |
| Built guide HTML | fenced code and figcaption | docs page body plus `Extract` of a `src=` target, caption only when the checker compared it | Yes | ✓ FLOWING |
| `llms.txt` / per-page `.md` | title, description, body | loaded page tree (`emit.go`) | Yes | ✓ FLOWING |
| Search index | page titles and text | the same page tree written by the build | Yes | ✓ FLOWING |
| API reference pages | README body | `modules/*/README.md` via `loadModules` | Yes | ✓ FLOWING |
### Behavioral Spot-Checks
| Behavior | Command | Result | Status |
|----------|---------|--------|--------|
| Real tree is clean | `summer docs:build --check` on a `git archive HEAD` copy | no problems found | ✓ PASS |
| Page-side drift refused | add a line to the bonfire fence in installation.md | `snippet: body differs … (run: summer docs:sync)` | ✓ PASS |
| Source-side drift refused + sync repairs | edit the ExampleCall description | refused; `docs:sync` updated 2 snippets; then clean | ✓ PASS |
| Unknown identifier (docs and README) | `lagoon.NoSuchThing` | `identifier: … does not exist in modules/lagoon` | ✓ PASS |
| Broken link and anchor | `../nope.md`, `#no-such-anchor` | both refused | ✓ PASS |
| Unknown command | `summer no:such`, `./bin/acme nope:cmd` | both refused | ✓ PASS |
| Go fence without src= | top-level ```go | refused | ✓ PASS |
| Forbidden name | planted in page | `forbidden: consuming-application name in output` | ✓ PASS |
| CR-01 callout src= fence with BOGUS body | `> ```go src=…#ExampleCall` | passes --check; build publishes BOGUS under the source caption | ✗ FAIL |
| WR-04 golang fence without src= | ```golang | passes | ✗ FAIL |
| WR-01 helper reached only by an Example without Output | src=…#neverRunHelper | passes | ✗ FAIL |
| WR-02 `//go:build ignore` source | src=modules/bonfire/broken.go#Broken | passes | ✗ FAIL |
| WR-03 wrong-case identifier | `lagoon.Openfromapp` | passes | ✗ FAIL |
| WR-05 env prefix / flag-first command | `FOO=1 summer no:such`, `summer --root . no:such2` | passes | ✗ FAIL |
| Walkthrough Docker tests | `go test -count=1 -v ./docs/examples/blog/...` | 7 top-level PASS in blog, sub-packages ok, 0 SKIP | ✓ PASS |
| Module docs Examples/regions | `go test -count=1 -run '^(Example\|TestDocs)' ./modules/...` | 57 PASS, 0 SKIP | ✓ PASS |
| --- | --- | --- | --- |
| Full phase gate, including scratch-tree plants of the old holes | `bash scripts/check-phase11.1.sh --all` | all stages passed, `phase11.1 all passed`, exit 0 | ✓ PASS |
| Docsite coverage at least 85% | `go test -count=1 -cover ./internal/docsite` | `coverage: 94.1% of statements` | ✓ PASS |
| Callout src=, golang fence, wrong-case ident, env-prefix and flag-first commands | self-test plants inside the gate (real-tree copy) | each `expect_refusal` returned; stage printed `phase11.1 self-test passed` | ✓ PASS |
| Named tests, skips refused | gate `--named` (`go test -json -count=1`) | `phase11.1 named passed` | ✓ PASS |
| Full module tests | gate `--go` (`go vet ./...` then `go test ./...`) | `phase11.1 go passed` (many packages `(cached)` after `--named` had already run the docs packages with `-count=1`) | ✓ PASS |
### Probe Execution
| Probe | Command | Result | Status |
|-------|---------|--------|--------|
| `scripts/check-phase11.1.sh` | `bash scripts/check-phase11.1.sh --all` | all 8 stages passed, `phase11.1 all passed`, exit 0 (the full `go test ./...` stage was served from Go's content-addressed test cache; the docs-critical packages were re-run uncached separately above) | PASS |
| --- | --- | --- | --- |
| `scripts/check-phase11.1.sh` | `bash scripts/check-phase11.1.sh --all` | exit 0, `phase11.1 all passed` | PASS |
### Requirements Coverage
| Requirement | Source Plan | Description | Status | Evidence |
|-------------|-------------|-------------|--------|----------|
| DOCS-01 | 01, 03, 04, 06 | Markdown pages with strict frontmatter in Winter sections; every module reachable via API page | ✓ SATISFIED | Truth 1 |
| DOCS-02 | 01, 02, 06 | self-contained site, docs:serve on loopback, no Node, only chroma/v2 added | ✓ SATISFIED (runtime UI pending human UAT) | Truth 2; serve tests |
| --- | --- | --- | --- | --- |
| DOCS-01 | 01, 03, 04, 06 | Markdown pages with strict frontmatter in Winter sections; every module reachable via an API page | ✓ SATISFIED | Truth 1 |
| DOCS-02 | 01, 02, 06 | self-contained site, docs:serve on loopback, no Node, only chroma/v2 added | ✓ SATISFIED | Truth 2. Search and theme interaction remain human UAT, as 11.1-VALIDATION.md already records. |
| DOCS-03 | 01, 06 | llms.txt, llms-full.txt, .md per page, sync test | ✓ SATISFIED | Truth 3 |
| DOCS-04 | 01–06 | every Go fence src=; a test fails on a missing or drifted snippet; Examples carry Output and run | ✗ BLOCKED (partial) | Content satisfies it (truth 4). Enforcement fails open (truths 6, 7). The REQUIREMENTS.md tick is premature. |
| DOCS-05 | 02, 06 | checkers fail on stale identifiers, broken links/anchors, unknown commands, forbidden names | ✗ BLOCKED (partial) | Truth 8. The REQUIREMENTS.md tick is premature. |
| DOCS-04 | 01–07 | every Go fence in docs/ has src=; a test fails on a missing or drifted snippet; Examples carry Output and run | ✓ SATISFIED | Truths 4, 6, 7 and G1–G7, G10. REQUIREMENTS.md still shows this box open and the traceability row as Gaps Found; this dispatch was not allowed to edit that file. The code satisfies the requirement. |
| DOCS-05 | 02, 06, 07 | checkers fail on stale identifiers, broken links and anchors, unknown commands, forbidden names | ✓ SATISFIED | Truth 8 and G8–G9. Same note: the REQUIREMENTS.md checkbox was left for a later edit and is not evidence. |
| DOCS-06 | 03, 04, 06 | concept map with checker-verified identifiers | ✓ SATISFIED | Truth 9 |
| DOCS-07 | 05, 06 | acme/blog walkthrough as a real in-root package verified by DOCS-04 | ✓ SATISFIED | Truth 5. Its fences are all top-level `go`/`yaml` src= fences, so they are unaffected by gaps 1–2 today. |
| DOCS-07 | 05, 06 | acme/blog walkthrough as a real in-root package | ✓ SATISFIED | Truth 5 |
| DOCS-08 | 02, 06 | CLAUDE.md documentation rule naming the checkers | ✓ SATISFIED | Truth 10 |
No orphaned requirements: REQUIREMENTS.md maps exactly DOCS-01..08 to Phase 11.1, and every ID is claimed by at least one plan.
No orphaned requirement. REQUIREMENTS.md maps exactly DOCS-01 through DOCS-08 to Phase 11.1, and every ID is claimed by at least one plan.
### Prohibitions (judgment-tier, non-authoritative LLM-judge verdicts; flagged for human review)
### Decision Coverage
All trackable CONTEXT.md decisions are honored by shipped artifacts. 18 honored, 18 total, none missing. Non-blocking.
### Prohibitions (judgment-tier)
**unverified-prohibition — human review recommended.** These verdicts are a non-authoritative LLM judge. They are not a silent pass. Autonomous completion is complete with 7 flagged prohibitions.
| # | Statement | Verdict | Evidence |
|---|-----------|---------|----------|
| 1 | The built site must not reference a third-party origin for assets (plans 01, 02) | holds | Built HTML has no external `src=`; CSS `url()` only points at vendored fonts; JS has no URLs. External `href`s are content links and the edit/source links (git host). |
| 2 | The theme must not load third-party fonts, icons or scripts, and has no analytics | holds | 8 woff2 fonts plus licences are vendored; the Lucide licence ships in assets |
| 3 | The docs must not describe an unprovided WinterCMS feature as available (plans 03, 04) | holds | Not provided table; frontend-and-ajax.md; llms_notes state "headless … no frontend themes" |
| 4 | The docs must not present an insecure setting as the default | holds (spot-checked only) | needs a human read of mail, push, search and serve pages |
| 5 | The walkthrough must not show a write path without a fill allow-list, or an admin controller without a permission | holds | `lagoon.Fill(post, post.Fillable(), …)` in models/post.go; `RequiredPermissions` in controllers/posts.go; parameterised `Where("slug = ?", slug)` |
| 6 | The gate must not count a skipped, filtered or zero-match run as a pass | holds | `go_json_named` refuses skip, no-tests and missing names |
| --- | --- | --- | --- |
| 1 | The built site must not reference a third-party origin for assets (plans 01, 02) | holds | Theme templates and assets were not modified by plan 07. The gate rebuilt the site. Prior inspection (no external `src=`, vendored font `url()`) still applies to those unchanged files. |
| 2 | The theme must not load third-party fonts, icons or scripts, and has no analytics | holds | Same unchanged theme tree. Licences ship under the theme assets. |
| 3 | The docs must not describe an unprovided WinterCMS feature as available (plans 03, 04) | holds | Not provided table re-read this pass; it links Frontend and AJAX. `docs/site.yaml` `llms_notes` still say the framework is headless. |
| 4 | The docs must not present an insecure setting as the default (plan 04) | holds (spot-check carried forward) | Mail, push, search and serve pages were not in the plan 07 diff. A full read is still a human item. |
| 5 | The walkthrough must not show a write without a fill allow-list, or an admin controller without a permission | holds | `lagoon.Fill(post, post.Fillable(), ...)` in `docs/examples/blog/models/post.go`; `RequiredPermissions` in `controllers/posts.go`. |
| 6 | The gate must not count a skipped, filtered or zero-match run as a pass | holds | `detect_json` exits on `skip`, `no tests to run`, zero passes and a missing required name. `--named` passed under that detector. |
| 7 | The site must not show a source caption over a code block the snippet checker did not compare (plan 07) | holds | `fenceAnnotator` matches the checked set. `TestFenceCaptionsOnlyVerified` and the acceptance figcaption cross-check ran inside the gate. |
### Test Quality Audit
| Test File | Linked Req | Active | Skipped | Circular | Assertion Level | Verdict |
| --- | --- | --- | --- | --- | --- | --- |
| `internal/docsite/violations_test.go` | DOCS-04, DOCS-05 | 81 directory cases plus runtime forbidden plants; each expects exactly one problem and a refused Build | 0 (`t.Skip` absent in `internal/docsite/*_test.go`) | No. Expected text is the planted `want.txt`, not output of the checker | Value | OK |
| `internal/docsite/snippet_test.go` `TestSnippetRootsAndBuild` | DOCS-04 | reachability and `Extract` error substrings | 0 | No. Oracle is `go/doc` and `go/build`, not the checker echoing itself | Behavioral | OK |
| `internal/docsite/checks_test.go` `TestCommandWord`, `TestIdentifierGoDocCaseSensitive`, `TestNestedFenceChecks` | DOCS-04, DOCS-05 | table and fixture assertions | 0 | No | Value | OK |
| `cmd/summer/phase11_1_acceptance_test.go` | DOCS-01..07 | SC1–SC5 plus `TestAcceptanceFenceScanner` | 0 in the named run (a skip would have failed the gate) | No. Scanner is independent of `docsite.collectFences` | Behavioral | OK |
| `scripts/check-phase11.1.sh` self-test | DOCS-04, DOCS-05 | plants on a copy of the real tree | n/a | No. The plant is a hand-written bad fence, not generated by the checker | Behavioral | OK |
**Disabled tests on requirements:** 0
**Circular patterns detected:** 0
**Insufficient assertions:** 0
### Anti-Patterns Found
| File | Line | Pattern | Severity | Impact |
|------|------|---------|----------|--------|
| (all phase-modified files) | – | TBD/FIXME/XXX | – | none found |
| (all phase-modified files) | – | TODO/HACK/PLACEHOLDER | – | none found |
| internal/docsite/render.go | 439–489 | line-based fence scanner diverges from goldmark AST | 🛑 Blocker | Gap 1 (CR-01) |
| internal/docsite/check_identifiers.go | 301 | case-insensitive `go doc` fallback | ⚠️ Warning | Gap 3 |
| internal/docsite/render.go | 207–232 | slug IDs not reserved against theme IDs (WR-06) | ⚠️ Warning (hardening) | The current output has no collisions: every theme ID appears once per page, and `# Search` on services/search.md is an H1 without an id. Not a gap. |
| docs/examples/blog/*.go | 1 | "Code generated … DO NOT EDIT" on edited files (WR-07) | ℹ️ Info | Intentional and documented: porting-a-plugin.md "What the scaffolder leaves to you" explains that `registry.gen.go` regeneration depends on the header. Tracked in `.planning/todos/pending/scaffold-generated-header-and-command-deps.md`. Not a gap. |
| --- | --- | --- | --- | --- |
| phase Go, the gate script, `docs/console/utilities.md`, `README.md` | — | TBD / FIXME / XXX | — | none |
| `internal/docsite` | — | `t.Skip` | — | none |
Review info items IN-01..IN-08 are hardening. None bears on a success criterion. IN-08 (spans with an unknown package prefix, `&pkg.X{}` forms) is outside the must-have's stated scope ("whose first segment is a discovered module").
### Advisory (New Scope, Unevidenced)
### Regression Check
New-scope findings from this re-verification with no deterministic evidence. Not blocking. They do not reopen a closed must-have.
- `go vet ./...` exits 0.
- `examples/hello` `TestTypedItemRoute` fails with `surf: config http.body_limits.default_bytes is required`. I reproduced the same failure on a `git archive` of 63bcbc3, the parent of the first 11.1 code commit (6dacddc). This is pre-existing and not a regression from this phase. It makes check-phase1.sh and check-phase4.sh fail and should be tracked separately.
| # | Finding | Category | Why Advisory |
| --- | --- | --- | --- |
| 1 | Identifier index still parses build-ignored Go files | other | Named in the plan 07 summary as out of scope. Not a prior gap. No failing test was run. |
### Human Verification Required
These items are listed in the frontmatter. They are search interaction, the theme with no flash and correct contrast, the no-JS layout, the usefulness of llms.txt and the .md files to AI agents, editorial and walkthrough followability, and resolution of the six judgment-tier prohibitions. Because gaps were found, the overall status is gaps_found; these items still need running in /gsd-verify-work.
Automated checks passed. These items are why status is `human_needed` rather than `passed`.
### 1. Search interaction
**Test:** Run `summer docs:serve`, search for a module name and a CLI command, use arrow keys and Enter.
**Expected:** Rows show section, title, heading and a marked excerpt; Enter opens the page; the live region announces the count.
**Why human:** Plan 02 and plan 06 backstop. No JS test runner.
### 2. Theme, contrast, no flash
**Test:** Cycle light, dark and system; reload; check a guide page and an API page at 1280px, 1024px and 375px.
**Expected:** No flash; the choice persists; UI-SPEC contrast holds.
**Why human:** Visual and first-paint timing.
### 3. No-JS layout
**Test:** Disable JavaScript and load a guide page below 1024px.
**Expected:** Sidebar above the content; search and theme buttons hidden.
**Why human:** Plan 02 backstop.
### 4. Raw files for an agent
**Test:** Read `llms.txt`, one page `.md` and `llms-full.txt`.
**Expected:** Spec shape and URLs that are useful without HTML.
**Why human:** Plan 01 backstop (`insufficient_spec`). unverified — held-out test recommended.
### 5. Editorial and walkthrough
**Test:** Read Setup, Backend, Database and Services, and follow `docs/setup/porting-a-plugin.md`.
**Expected:** WinterCMS tone, accurate to the READMEs, ending at `docs/examples/blog`.
**Why human:** Plans 03, 04 and 05 backstop.
### 6. Judgment-tier prohibitions
**Test:** Confirm the seven prohibition rows above.
**Expected:** Each holds.
**Why human:** unverified-prohibition — human review recommended.
### Gaps Summary
All three gaps share one root cause: the docs checkers **fail open** on Markdown and shell forms they do not model. They pass silently when they should refuse. None of them is exploited by the current content, which is fully verified. They matter because DOCS-04 and DOCS-05 are guard-rail requirements: the phase goal says every code example is compiled and tested, and CLAUDE.md now tells future changes to rely on `TestDocsTree` and `docs:build --check`.
1. **Fence discovery (BLOCKER, CR-01 + WR-04).** Switch snippet, policy and command fence collection from `scanFences` to the goldmark AST, or refuse non-top-level src= and Go fences. Normalise Go language aliases. Stop captioning unchecked README src= fences. Add planted fixtures and fix the acceptance scanner.
2. **Proof of execution (WR-01 + WR-02).** Only Examples with output are roots, and src= files must be in the default build.
3. **Identifier and command forms (WR-03 + WR-05).** Pass `go doc -c`, and parse env-prefix, flag-first and `go run ./cmd/summer` command forms.
All three are local to `internal/docsite` with small fixes, and fit one gap-closure plan that ends with planted-violation fixtures for each hole. No later milestone phase covers them. Phase 11.2 consumes the docs but does not harden the checkers, so none of the gaps is deferred.
No code gap remains from the previous report. Fence discovery, proof of execution, and the identifier and command forms now fail closed, with planted fixtures and a green `scripts/check-phase11.1.sh --all`. Nothing in a later milestone phase was needed to defer them. The open work is human UAT already scheduled in `11.1-VALIDATION.md`, plus explicit resolution of the judgment-tier prohibitions. Do not open another gap-closure plan for those.
---
_Verified: 2026-09-30T23:06:42Z_
_Verifier: Claude (gsd-verifier)_
_Verified: 2026-10-01T07:31:53Z_
_Verifier: the agent (gsd-verifier)_