35 KiB
phase, verified, status, score, covered_files, covered_digest, behavior_unverified, overrides_applied, gaps, human_verification
| phase | verified | status | score | covered_files | covered_digest | behavior_unverified | overrides_applied | gaps | human_verification | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 11.1-summercms-documentation-for-humans-and-ai-agents | 2026-09-30T23:06:42Z | gaps_found | 10/13 must-haves verified |
|
v2:sha256:c68da6c413348d7d4d0d272df3d007aa82e2f9ca5aa9f9ab79eefc797d2cb7d0 | 0 | 0 |
|
|
Phase 11.1: SummerCMS documentation for humans and AI agents Verification Report
Phase Goal: SummerCMS has a WinterCMS-style documentation set that serves both humans and AI agents. The Markdown source lives in summercms.go/docs/ and a summer CLI command builds it into a static site with sidebar navigation, search, llms.txt/llms-full.txt and a raw .md per page. Every code example compiles and is tested, and a "Coming from WinterCMS" map plus an acme/blog porting walkthrough cover the migration path. It documents the framework as it stands after Phase 11 and never names a consuming application.
Verified: 2026-09-30T23:06:42Z
Status: gaps_found
Re-verification: No, initial verification
Verdict in one paragraph
The documentation set exists and is correct today. There are 50 guide pages in 8 Winter-mirroring sections and 22 API reference pages ingested from the module READMEs. The build writes HTML, a .md per page, llms.txt and llms-full.txt, with no Node toolchain. Every one of the 106 Go fences in docs/ is a src= copy that is byte-checked against source that go test ./... compiles and runs. The concept map and the Docker-tested acme/blog walkthrough are in place, and no consuming application is named anywhere. The gaps are in the enforcement that DOCS-04 and DOCS-05 promise: "a test fails on a missing or drifted snippet" and "checkers fail on stale identifiers and unknown commands". I reproduced every review finding the orchestrator asked about on a git archive HEAD copy of the real tree, and each checker fails open. The worst case is CR-01: a src= fence inside a callout is published with a "verified source" caption over code nobody checked. No current page exploits any of these holes, so these gaps concern guard-rail correctness, not content correctness. Each fix is small and local to internal/docsite.
Goal Achievement
Observable Truths
| # | Truth | Status | Evidence |
|---|---|---|---|
| 1 | SC1: docs/ holds Markdown pages with frontmatter in Setup, Architecture, Plugins, Backend, Database, Services, Console, API reference, and every framework module is reachable from the sidebar | ✓ VERIFIED | docs/site.yaml lists the 8 sections in that order; 50 guide pages; built site has api/<m>.html for all 22 top-level modules. TestPhase11_1Acceptance/SC1 PASS (run by me, -count=1). Sub-packages (attach, typesense, centrifugo) are documented inside their parent README, not as separate pages (see Info). |
| 2 | SC2: a summer command builds a self-contained static site with sidebar, TOC, prev/next, client-side search and dark mode, no Node toolchain; only new dep is approved |
✓ VERIFIED (static); interaction to human UAT | summer docs:build --out <scratch> wrote 72 pages; theme markers asserted by SC2 PASS; no exec of node/npm/npx (SC2 AST scan); git log of 11.1 commits shows go.mod gained only alecthomas/chroma/v2 + indirect dlclark/regexp2/v2 (goldmark predates the phase, from 04-03). Built HTML/CSS/JS loads nothing from a third-party origin (fonts vendored under /assets/fonts). Search interaction and theme flash are routed to human verification. |
| 3 | SC3/DOCS-03: llms.txt, llms-full.txt and a clean .md per page, with a test asserting sync with the page tree |
✓ VERIFIED | Built output inspected: llms.txt has # SummerCMS, a blockquote, H2 link lists; setup/installation.md starts # Installation / > description, no frontmatter. TestDocsAIOutputsInSync and SC3 PASS. |
| 4 | SC4 (current content): every Go example in a docs/ page is compiled and run by go test ./... |
✓ VERIFIED | 106 go fences, 0 without `src=`, none inside blockquotes, no golang fences. All src= targets are in root-module packages (SC4 go-list check). AST scan (go/doc): no Example in any example*_test.go or docs/examples test lacks // Output:. No src= target has a build constraint. `go test -count=1 -run '^(Example |
| 5 | SC5 / DOCS-07: the concept map and the acme/blog walkthrough exist, and the walkthrough's code is verified under SC4 | ✓ VERIFIED | docs/setup/porting-a-plugin.md (829 lines) covers plugin registration, model, migrations (plus a rollback column), routes, admin controller and console command; its fences are src= copies from docs/examples/blog (in-root package, no go.mod). go test -count=1 -v ./docs/examples/blog/... ran TestMigrateUpAndRollback, TestPostsRouteAgainstDatabase, TestPublishCommandAgainstDatabase and TestScaffoldLayout on Docker, all PASS, none skipped. SC5 PASS. |
| 6 | DOCS-04 enforcement: a test fails on a missing or drifted snippet; every Go fence must carry src= | ✗ FAILED | Top-level fences are enforced: drift in the page and drift in the source both refuse, and docs:sync repairs them (mutation-tested). Fences inside callouts and blockquotes are never checked but are still captioned as verified source (CR-01, reproduced on real tree copy). ```golang bypasses the src= policy (WR-04, reproduced). README src= fences are captioned without being checked. |
| 7 | D-07 proof of execution: src= must name compiled source, and code no Test or Example runs is a snippet problem | ✗ FAILED (partial) | An Example without Output counts as a root (WR-01, reproduced). //go:build ignore files pass as compiled (WR-02, reproduced). Not exploited on the real tree. |
| 8 | DOCS-05: checkers fail on stale identifiers, broken links/anchors, unknown command names and forbidden names | ✗ FAILED (partial) | Links, anchors, forbidden names, unknown identifiers and plain summer x/./bin/app x forms all refuse (mutation-tested). Wrong-case identifiers pass via the go doc fallback without -c (WR-03: lagoon.Openfromapp passed). FOO=1 summer no:such and summer --root . no:such2 pass (WR-05). Not exploited: all 704 module spans resolve with go doc -c. |
| 9 | DOCS-06: "Coming from WinterCMS" maps Winter concepts to SummerCMS, with checker-verified identifiers and explicit Not provided rows | ✓ VERIFIED | Concept table covers Plugin.php, $require, permissions/navigation, version.yaml, Eloquent, fields/columns.yaml, backend controllers, routes.php, middleware, config, lang, events, App::make, artisan, queues, scheduler, mail, settings, broadcasting, Scout, the HTTP client. A Not provided table covers CMS pages/themes, components, AJAX/Snowboard, media manager, import/export, reorder, collections, behaviours, cache and session, linking frontend-and-ajax.md. All pkg.Ident spans resolve case-sensitively. |
| 10 | DOCS-08: CLAUDE.md records the docs update rule and names the checkers | ✓ VERIFIED | CLAUDE.md ## Documentation (HEAD, clean worktree) has the rule for "affected pages under docs/", names go test ./cmd/summer -run TestDocsTree and summer docs:build --check, and notes that config-key checking is deferred. The gate's --claude stage passes. |
| 11 | D-11: no page, snippet or built output names a consuming application | ✓ VERIFIED | grep -rliE over docs/, all module READMEs, the root README and the full built site finds nothing, except the checker's own pattern in check_forbidden.go. A planted name is refused (mutation-tested). |
| 12 | Phase-end tests: docsite coverage ≥85%, a planted fixture per rule, and acceptance subtests SC1–SC5 | ✓ VERIFIED | go test -cover ./internal/docsite gives 94.8%. TestPlantedViolations has 68 passing subtests. TestPhase11_1Acceptance SC1–SC5 PASS. |
| 13 | The gate: check-phase11.1.sh --all is green and --named refuses skipped or zero-match runs |
✓ VERIFIED | I ran bash scripts/check-phase11.1.sh --all. Every stage passed (preconditions, deps, self-test, docs, forbidden, claude, named, go) and it printed phase11.1 all passed, exit 0. go_json_named refuses skips and "no tests to run" (script lines 172–211). |
Score: 10/13 truths verified (0 present, behavior-unverified).
Five backstop truths from the plans, which cover editorial usefulness, followability of the walkthrough, search rows and keys, no theme flash, contrast and the no-JS layout, are non-inferable. They are routed to human verification and not scored.
Required Artifacts
| Artifact | Expected | Status | Details |
|---|---|---|---|
internal/docsite/docsite.go |
Build, Check, Sync entry points | ✓ VERIFIED | Called from cmd/summer/docs.go |
internal/docsite/load.go |
strict frontmatter, README ingestion | ✓ VERIFIED | loadModules discovers modules dynamically |
internal/docsite/render.go |
goldmark with shared slug IDs | ✓ VERIFIED (with gap) | parser.WithIDs; scanFences is the root of gap 1 |
internal/docsite/emit.go |
html, .md, llms*, search index | ✓ VERIFIED | outputs inspected |
internal/docsite/snippet.go |
src= parse, confine, extract, drift | ✓ VERIFIED (with gaps) | drift works top-level; gaps 1 and 2 |
internal/docsite/check_identifiers.go |
identifier index with go doc fallback | ✓ VERIFIED (with gap) | gap 3 (case) |
internal/docsite/check_links.go |
links and anchors | ✓ VERIFIED | mutation-tested |
internal/docsite/check_commands.go |
command names from toolCommands and runtime set | ✓ VERIFIED (with gap) | gap 3 (forms) |
internal/docsite/check_forbidden.go |
forbidden names over sources and outputs | ✓ VERIFIED | mutation-tested |
internal/docsite/highlight.go |
chroma/v2 NodeRenderer | ✓ VERIFIED | tok-* classes in built HTML |
internal/docsite/serve.go |
loopback server, 404, rebuild | ✓ VERIFIED | serve tests pass; flag --allow-remote registered |
cmd/summer/docs.go |
docs:build, docs:sync, docs:serve | ✓ VERIFIED | in toolCommands(); binary built and run |
docs/setup/coming-from-wintercms.md |
concept map | ✓ VERIFIED | |
docs/setup/porting-a-plugin.md |
walkthrough | ✓ VERIFIED | |
docs/examples/blog/* |
in-root verified plugin | ✓ VERIFIED | Docker tests ran |
internal/docsite/violations_test.go |
TestPlantedViolations | ✓ VERIFIED | has no fixtures for gaps 1–3 |
cmd/summer/phase11_1_acceptance_test.go |
SC1–SC5 | ✓ VERIFIED (with gap) | scanner shares the blind spot in gap 1 |
scripts/check-phase11.1.sh |
--self-test, --named, --all | ✓ VERIFIED | ran --all |
Key Link Verification
| From | To | Via | Status |
|---|---|---|---|
| cmd/summer/docs.go | internal/docsite | docsite.Build / Sync / Check | WIRED |
| cmd/summer/docs.go | check_commands.go | docsCommands() → toolCommands() + module command constructors | WIRED |
| render.go | goldmark | parser.WithIDs(slug IDs), shared with check_links | WIRED |
| highlight.go | chroma/v2 | lexers + tokenise → tok-* | WIRED |
| docs/setup/installation.md | modules/bonfire/example_test.go | src=…#ExampleCall | WIRED (drift is detected) |
| docs/setup/porting-a-plugin.md | docs/examples/blog/* | 15+ src= fences | WIRED |
| docs/examples/blog/scaffold_layout_test.go | internal/build | build.Make* | WIRED (TestScaffoldLayout PASS) |
| docs/setup/coming-from-wintercms.md | docs/services/frontend-and-ajax.md | Not provided rows link it | WIRED |
| scripts/check-phase11.1.sh | TestPlantedViolations | --self-test / --named | WIRED |
Behavioral Spot-Checks
| Behavior | Command | Result | Status |
|---|---|---|---|
| Real tree is clean | summer docs:build --check on a git archive HEAD copy |
no problems found | ✓ PASS |
| Page-side drift refused | add a line to the bonfire fence in installation.md | snippet: body differs … (run: summer docs:sync) |
✓ PASS |
| Source-side drift refused + sync repairs | edit the ExampleCall description | refused; docs:sync updated 2 snippets; then clean |
✓ PASS |
| Unknown identifier (docs and README) | lagoon.NoSuchThing |
identifier: … does not exist in modules/lagoon |
✓ PASS |
| Broken link and anchor | ../nope.md, #no-such-anchor |
both refused | ✓ PASS |
| Unknown command | summer no:such, ./bin/acme nope:cmd |
both refused | ✓ PASS |
| Go fence without src= | top-level ```go | refused | ✓ PASS |
| Forbidden name | planted in page | forbidden: consuming-application name in output |
✓ PASS |
| CR-01 callout src= fence with BOGUS body | > ```go src=…#ExampleCall |
passes --check; build publishes BOGUS under the source caption | ✗ FAIL |
| WR-04 golang fence without src= | ```golang | passes | ✗ FAIL |
| WR-01 helper reached only by an Example without Output | src=…#neverRunHelper | passes | ✗ FAIL |
WR-02 //go:build ignore source |
src=modules/bonfire/broken.go#Broken | passes | ✗ FAIL |
| WR-03 wrong-case identifier | lagoon.Openfromapp |
passes | ✗ FAIL |
| WR-05 env prefix / flag-first command | FOO=1 summer no:such, summer --root . no:such2 |
passes | ✗ FAIL |
| Walkthrough Docker tests | go test -count=1 -v ./docs/examples/blog/... |
7 top-level PASS in blog, sub-packages ok, 0 SKIP | ✓ PASS |
| Module docs Examples/regions | go test -count=1 -run '^(Example|TestDocs)' ./modules/... |
57 PASS, 0 SKIP | ✓ PASS |
Probe Execution
| Probe | Command | Result | Status |
|---|---|---|---|
scripts/check-phase11.1.sh |
bash scripts/check-phase11.1.sh --all |
all 8 stages passed, phase11.1 all passed, exit 0 (the full go test ./... stage was served from Go's content-addressed test cache; the docs-critical packages were re-run uncached separately above) |
PASS |
Requirements Coverage
| Requirement | Source Plan | Description | Status | Evidence |
|---|---|---|---|---|
| DOCS-01 | 01, 03, 04, 06 | Markdown pages with strict frontmatter in Winter sections; every module reachable via API page | ✓ SATISFIED | Truth 1 |
| DOCS-02 | 01, 02, 06 | self-contained site, docs:serve on loopback, no Node, only chroma/v2 added | ✓ SATISFIED (runtime UI pending human UAT) | Truth 2; serve tests |
| DOCS-03 | 01, 06 | llms.txt, llms-full.txt, .md per page, sync test | ✓ SATISFIED | Truth 3 |
| DOCS-04 | 01–06 | every Go fence src=; a test fails on a missing or drifted snippet; Examples carry Output and run | ✗ BLOCKED (partial) | Content satisfies it (truth 4). Enforcement fails open (truths 6, 7). The REQUIREMENTS.md tick is premature. |
| DOCS-05 | 02, 06 | checkers fail on stale identifiers, broken links/anchors, unknown commands, forbidden names | ✗ BLOCKED (partial) | Truth 8. The REQUIREMENTS.md tick is premature. |
| DOCS-06 | 03, 04, 06 | concept map with checker-verified identifiers | ✓ SATISFIED | Truth 9 |
| DOCS-07 | 05, 06 | acme/blog walkthrough as a real in-root package verified by DOCS-04 | ✓ SATISFIED | Truth 5. Its fences are all top-level go/yaml src= fences, so they are unaffected by gaps 1–2 today. |
| DOCS-08 | 02, 06 | CLAUDE.md documentation rule naming the checkers | ✓ SATISFIED | Truth 10 |
No orphaned requirements: REQUIREMENTS.md maps exactly DOCS-01..08 to Phase 11.1, and every ID is claimed by at least one plan.
Prohibitions (judgment-tier, non-authoritative LLM-judge verdicts; flagged for human review)
| # | Statement | Verdict | Evidence |
|---|---|---|---|
| 1 | The built site must not reference a third-party origin for assets (plans 01, 02) | holds | Built HTML has no external src=; CSS url() only points at vendored fonts; JS has no URLs. External hrefs are content links and the edit/source links (git host). |
| 2 | The theme must not load third-party fonts, icons or scripts, and has no analytics | holds | 8 woff2 fonts plus licences are vendored; the Lucide licence ships in assets |
| 3 | The docs must not describe an unprovided WinterCMS feature as available (plans 03, 04) | holds | Not provided table; frontend-and-ajax.md; llms_notes state "headless … no frontend themes" |
| 4 | The docs must not present an insecure setting as the default | holds (spot-checked only) | needs a human read of mail, push, search and serve pages |
| 5 | The walkthrough must not show a write path without a fill allow-list, or an admin controller without a permission | holds | lagoon.Fill(post, post.Fillable(), …) in models/post.go; RequiredPermissions in controllers/posts.go; parameterised Where("slug = ?", slug) |
| 6 | The gate must not count a skipped, filtered or zero-match run as a pass | holds | go_json_named refuses skip, no-tests and missing names |
Anti-Patterns Found
| File | Line | Pattern | Severity | Impact |
|---|---|---|---|---|
| (all phase-modified files) | – | TBD/FIXME/XXX | – | none found |
| (all phase-modified files) | – | TODO/HACK/PLACEHOLDER | – | none found |
| internal/docsite/render.go | 439–489 | line-based fence scanner diverges from goldmark AST | 🛑 Blocker | Gap 1 (CR-01) |
| internal/docsite/check_identifiers.go | 301 | case-insensitive go doc fallback |
⚠️ Warning | Gap 3 |
| internal/docsite/render.go | 207–232 | slug IDs not reserved against theme IDs (WR-06) | ⚠️ Warning (hardening) | The current output has no collisions: every theme ID appears once per page, and # Search on services/search.md is an H1 without an id. Not a gap. |
| docs/examples/blog/*.go | 1 | "Code generated … DO NOT EDIT" on edited files (WR-07) | ℹ️ Info | Intentional and documented: porting-a-plugin.md "What the scaffolder leaves to you" explains that registry.gen.go regeneration depends on the header. Tracked in .planning/todos/pending/scaffold-generated-header-and-command-deps.md. Not a gap. |
Review info items IN-01..IN-08 are hardening. None bears on a success criterion. IN-08 (spans with an unknown package prefix, &pkg.X{} forms) is outside the must-have's stated scope ("whose first segment is a discovered module").
Regression Check
go vet ./...exits 0.examples/helloTestTypedItemRoutefails withsurf: config http.body_limits.default_bytes is required. I reproduced the same failure on agit archiveof63bcbc3, the parent of the first 11.1 code commit (6dacddc). This is pre-existing and not a regression from this phase. It makes check-phase1.sh and check-phase4.sh fail and should be tracked separately.
Human Verification Required
These items are listed in the frontmatter. They are search interaction, the theme with no flash and correct contrast, the no-JS layout, the usefulness of llms.txt and the .md files to AI agents, editorial and walkthrough followability, and resolution of the six judgment-tier prohibitions. Because gaps were found, the overall status is gaps_found; these items still need running in /gsd-verify-work.
Gaps Summary
All three gaps share one root cause: the docs checkers fail open on Markdown and shell forms they do not model. They pass silently when they should refuse. None of them is exploited by the current content, which is fully verified. They matter because DOCS-04 and DOCS-05 are guard-rail requirements: the phase goal says every code example is compiled and tested, and CLAUDE.md now tells future changes to rely on TestDocsTree and docs:build --check.
- Fence discovery (BLOCKER, CR-01 + WR-04). Switch snippet, policy and command fence collection from
scanFencesto the goldmark AST, or refuse non-top-level src= and Go fences. Normalise Go language aliases. Stop captioning unchecked README src= fences. Add planted fixtures and fix the acceptance scanner. - Proof of execution (WR-01 + WR-02). Only Examples with output are roots, and src= files must be in the default build.
- Identifier and command forms (WR-03 + WR-05). Pass
go doc -c, and parse env-prefix, flag-first andgo run ./cmd/summercommand forms.
All three are local to internal/docsite with small fixes, and fit one gap-closure plan that ends with planted-violation fixtures for each hole. No later milestone phase covers them. Phase 11.2 consumes the docs but does not harden the checkers, so none of the gaps is deferred.
Verified: 2026-09-30T23:06:42Z Verifier: Claude (gsd-verifier)