--- phase: 11.1-summercms-documentation-for-humans-and-ai-agents plan: 04 subsystem: docs tags: [docs, wintercms, backend, database, services, examples, lagoon, conga, lighthouse, beachcomber, flare, tide, cabana] requires: - phase: 11.1-03 provides: "Setup, Architecture, Plugins and Console sections, the concept map, requiredPages and the example-file patterns" - phase: 11-08 provides: "lagoon.Transaction and lagoon.AfterCommit gates (nested calls bound to the parent handle, foreign transactions fail closed)" provides: - "docs/backend: admin-controllers, forms, lists-and-filters, relation-manager, users-and-permissions, settings, partials-and-widgets, admin-spa" - "docs/database: models, migrations, queries-and-pagination, relations, casts-and-validation, attachments, transactions" - "docs/services: configuration, events, routing, rate-limiting, authentication, oauth-server, mail, localization, storage, outbound-http, jobs, realtime, push, search, parity-testing, frontend-and-ajax" - "site.yaml sections in the D-08 order setup, architecture, plugins, backend, database, services, console, api, asserted by TestDocsRequiredPages" - "Runnable Examples in 19 packages and database-backed TestDocs* regions in conga, lagoon, lighthouse and beachcomber through test-only harness exports" - "Concept map rows linked to their guide pages; not-provided rows linked to the Frontend and AJAX page" affects: [11.1-05, 11.1-06] estimate_ref: "tokens 150000, tasks 3, confidence low" actuals: tokens: 71500 tasks: 3 commits: 4 plan_head_before: 69dd5764bb6157e53ef0559879df6a68c85a26dd plan_head_after: 44bd1446f572c92b31e16f7f6153349614af2c12 tech-stack: added: [] patterns: - "Database-bound snippets are docs:start regions in a helper that a TestDocs* test calls with a database from an exported test-only harness wrapper (export_docs_test.go: conga.DocsApp, lagoon.DocsDB, lighthouse.DocsEnv, beachcomber.DocsApp)" - "A declaration shown by #Ident must be reachable from a Test or Example; TestDocsDeclarations tests call the methods GORM or the framework would call (hooks, Migrations, Permissions, Searchable methods)" - "Docs examples never register a global driver or engine: the registered-name lists are asserted by package tests, so a toy engine is installed with a test-only hook (beachcomber.DocsUseEngine) and driver-specific examples live in the driver's package" - "YAML shown on Backend pages is a yaml src= copy of modules/cabana/testdata/docs fixtures that an Example compiles" key-files: created: - docs/backend/admin-controllers.md - docs/backend/forms.md - docs/backend/lists-and-filters.md - docs/backend/relation-manager.md - docs/backend/users-and-permissions.md - docs/backend/settings.md - docs/backend/partials-and-widgets.md - docs/backend/admin-spa.md - docs/database/models.md - docs/database/migrations.md - docs/database/queries-and-pagination.md - docs/database/relations.md - docs/database/casts-and-validation.md - docs/database/attachments.md - docs/database/transactions.md - docs/services/configuration.md - docs/services/events.md - docs/services/routing.md - docs/services/rate-limiting.md - docs/services/authentication.md - docs/services/oauth-server.md - docs/services/mail.md - docs/services/localization.md - docs/services/storage.md - docs/services/outbound-http.md - docs/services/jobs.md - docs/services/realtime.md - docs/services/push.md - docs/services/search.md - docs/services/parity-testing.md - docs/services/frontend-and-ajax.md - modules/conga/example_test.go - modules/conga/export_docs_test.go - modules/lagoon/example_test.go - modules/lagoon/export_docs_test.go - modules/lagoon/attach/example_test.go - modules/compass/example_test.go - modules/surf/example_test.go - modules/wire/example_test.go - modules/bouncer/example_test.go - modules/wristband/example_test.go - modules/postcard/example_test.go - modules/phrasebook/example_test.go - modules/cabana/example_test.go - modules/cabana/example_controller_test.go - modules/cabana/testdata/docs/ - modules/fetchguard/example_test.go - modules/lighthouse/example_test.go - modules/lighthouse/export_docs_test.go - modules/lighthouse/centrifugo/example_test.go - modules/flare/example_test.go - modules/beachcomber/example_test.go - modules/beachcomber/export_docs_test.go - modules/beachcomber/typesense/example_test.go - modules/tide/example_test.go - modules/tide/testdata/docs/posts-spec.yaml - .planning/todos/pending/lagoon-validate-min-message.md - .planning/todos/pending/lagoon-readme-after-commit-callback-order.md modified: - docs/site.yaml - docs/index.md - docs/setup/coming-from-wintercms.md - modules/festival/example_test.go - cmd/summer/docs_test.go key-decisions: - "The transactions page follows lagoon at HEAD (after 9526b6b and 93c735b), not only 11-08-SUMMARY: a nested lagoon.Transaction must get the outer tx and errors on a root handle; AfterCommit inside a plain GORM transaction warns and skips" - "The OnDatabase example registers its callback After(gorm:create).Before(gorm:commit_or_rollback_transaction): the lagoon README's After(gorm:after_create) form sorts after lagoon:after_commit and silently drops the work (reproduced by TestDocsOnDatabase, logged as a todo)" - "Pivot rows are read in order through an explicit join; the page warns that ordering a many2many Preload by a pivot column fails in GORM" - "Validation pages describe lagoon as it is, including that a numeric min failure is reported with the max message; the fix is a logged todo, not a module change" - "Backend YAML comes from new testdata/docs fixtures compiled by ExampleCompileList, ExampleCompileForm and ExampleActivate rather than hand-written blocks" - "wristband snippets come only from modules/wristband/example_test.go and set their own Issuer and Resource; the default resource URL is never quoted" - "Docs examples do not register global drivers or engines, because lighthouse and beachcomber tests assert the registered-name lists" patterns-established: - "export_docs_test.go per database-backed module exposes its harness to the external example package" - "TestDocsRequiredPages checks the site.yaml section order against sectionOrder" requirements-completed: [DOCS-01, DOCS-04, DOCS-06] coverage: - id: D1 description: "Backend, Database and Services sections are in the sidebar in the D-08 order and all 31 new pages build as .html and .md" requirement: DOCS-01 verification: - kind: unit ref: "cmd/summer/docs_test.go#TestDocsRequiredPages" status: pass - kind: unit ref: "cmd/summer/docs_test.go#TestDocsAIOutputsInSync" status: pass - kind: unit ref: "cmd/summer/docs_test.go#TestDocsBuildRealTree" status: pass human_judgment: false - id: D2 description: "Every Go fence on the new pages is a src= copy of an Example with // Output: or a docs region that a TestDocs* test runs against the module's Postgres harness" requirement: DOCS-04 verification: - kind: integration ref: "modules/conga/example_test.go#TestDocsDispatch" status: pass - kind: integration ref: "modules/lagoon/example_test.go#TestDocsTransactions" status: pass - kind: integration ref: "modules/lighthouse/example_test.go#TestDocsBroadcast" status: pass - kind: integration ref: "modules/beachcomber/example_test.go#TestDocsSearch" status: pass - kind: other ref: "go run ./cmd/summer docs:build --check" status: pass human_judgment: false - id: D3 description: "The concept map links every guide page and the not-provided rows link the Frontend and AJAX page, which states that CMS pages, themes, components, the AJAX framework and Snowboard are not provided" requirement: DOCS-06 verification: - kind: unit ref: "cmd/summer/docs_test.go#TestDocsTree" status: pass - kind: other ref: "grep -c 'frontend-and-ajax.md' docs/setup/coming-from-wintercms.md (4)" status: pass human_judgment: false - id: D4 description: "No page or output names a consuming application; phase 10 and 11 hygiene gates pass over the new example files" verification: - kind: other ref: "scripts/check-phase11.1.sh --docs and --forbidden; scripts/check-phase10.sh --hygiene; scripts/check-phase11.sh --hygiene" status: pass human_judgment: false - id: D5 description: "Pages are accurate to the modules, state the secure defaults (mandatory STARTTLS, allowlisted https push hosts, SQL re-gate of SearchIDs, secret-checked subscribe proxy) and read well for a WinterCMS developer" verification: [] human_judgment: true rationale: "Backstop truth in the plan: accuracy against the READMEs and usefulness are a reader's judgment" duration: 47min completed: 2026-09-30 status: complete --- # Phase 11.1 Plan 04: Framework content B (Backend, Database, Services) Summary **Thirty-one new guide pages cover the admin backend, the GORM data layer with lagoon transactions and after-commit work, and every framework service from configuration to Web Push, search and parity testing, plus a Frontend and AJAX (not provided) page. Every Go block is an Example or a docs region that `go test` runs, the database ones against each module's Postgres container.** ## Performance - **Duration:** about 47 min - **Started:** 2026-09-30T20:31:46Z - **Completed:** 2026-09-30T21:19Z - **Tasks:** 3 - **Files modified:** 68 ## Accomplishments - **Services section and Jobs tracer:** `docs/services/jobs.md` shows a typed job, its registration through `pact.HasJobs`, dispatch inside the caller's transaction, the `summer_jobs` statuses, progress and cancellation and the worker commands. `TestDocsDispatch` starts a real worker on the conga harness and waits for the dispatched job to complete its row. - **Database section:** models (Fill, Hidden, hooks, the models-leaf rule), migrations (gormigrate sets, history tables, rollback), queries and pagination (allow-listed `OrderBy`, `Paginate`), relations (pivot models, soft-delete cascades), casts and validation (Jsonable, Encrypted with key rotation, Laravel rule strings), attachments (system_files, thumbnails, two-phase delete) and transactions (AfterCommit rules table, savepoints, the root-handle refusal, `OnDatabase` callbacks). Six `TestDocs*` tests run the regions on the lagoon harness. - **Core Services:** configuration layers, events (Fire, Collect, UntilHandled with new festival Examples), routing with an auth group built from a bouncer guard, rate limiting and trusted proxies, authentication (tokens, refresh, blacklist, guards, bcrypt), the OAuth server (no default resource quoted), mail (STARTTLS mandatory by default, `starttls`/`none` marked development-only) and localization (CLDR plurals with post/posty/postów). - **Backend section:** admin controllers, forms, lists and filters, relation manager, users and permissions, settings, partials and widgets and the admin SPA. YAML blocks are copies of new `modules/cabana/testdata/docs` fixtures that `ExampleCompileList`, `ExampleCompileForm` and `ExampleActivate` compile. - **Phase 11 services as shipped:** realtime (authorizers on every subscribe, `lighthouse.Mount` surfaces, broadcasts enqueued in the write transaction and published after commit, `WithoutBroadcasting` plus `Emit`, the Centrifugo proxy answering 200 with a constant-time secret check), Web Push (https only, allowlisted hosts, no redirects, VAPID key handling), search (after-commit sync, kill-switch gate, stale candidates re-checked in SQL, Typesense requests) and parity testing (record, replay, masked diffs, broadcast goldens). - **Frontend and AJAX (not provided):** states the headless model and what is not provided, and routes readers to routing, authentication and realtime. The concept map now links a guide page on every row that has one. - `TestDocsRequiredPages` covers 49 pages and asserts the section order `setup, architecture, plugins, backend, database, services, console, api`. ## Task Commits 1. **Task 1: Tracer, the Services section with a verified Jobs page:** `9d37d56` (feat) 2. **Task 2: Database section and the core Services pages:** `efb35a2` (feat) 3. **Todos for the lagoon gaps found in Task 2:** `f7dfe68` (docs) 4. **Task 3: Backend section, remaining Services pages and concept-map links:** `44bd144` (feat) **Plan metadata:** the docs(11.1-04) commit that adds this file ## Files Created/Modified See `key-files` in the frontmatter. ## Decisions Made See `key-decisions` in the frontmatter. ## Deviations from Plan ### Auto-fixed Issues **1. [Rule 1 - Bug] The lagoon README's after-commit callback pattern drops its work** - **Found during:** Task 2 - **Issue:** A create callback registered `After("gorm:after_create")` that calls `lagoon.AfterCommit` is sorted after `lagoon:after_commit`, so its buffered work never runs. `TestDocsOnDatabase` reproduced it. - **Fix:** The docs region registers `After("gorm:create").Before("gorm:commit_or_rollback_transaction")` and the Transactions page warns about the ordering. The README (read-only for this plan) is logged in `.planning/todos/pending/lagoon-readme-after-commit-callback-order.md`. - **Commit:** efb35a2, f7dfe68 **2. [Rule 1 - Accuracy] Pivot ordering through Preload fails** - **Found during:** Task 2 - **Issue:** Ordering a many2many `Preload` by a pivot column fails with "missing FROM-clause entry", although the lagoon comment and README describe it. - **Fix:** The Relations region reads through an explicit join; the page carries a warning; the README claim is in the same todo. - **Commit:** efb35a2 **3. [Rule 1 - Accuracy] Numeric min failures use the max message** - **Found during:** Task 2 - **Issue:** `lagoon.Validate` answers a `min` failure on an integer field with "may not be greater than ." (empty limit). - **Fix:** The example uses `max`; the page has a NOTE describing the behaviour; the fix is logged in `.planning/todos/pending/lagoon-validate-min-message.md`. No module change. - **Commit:** efb35a2, f7dfe68 **4. [Rule 3 - Blocking] Docs examples changed the registered driver and engine lists** - **Found during:** Task 3 (full `go test ./...` run) - **Issue:** Registering an `acme-memory` search engine from an example's `init`, and importing the centrifugo driver into the lighthouse example package, changed the registered-name lists that `TestServiceSetup` (beachcomber, read-only `sync_test.go`) and `TestFromSelectsDriver` (lighthouse) assert. - **Fix:** The toy engine is installed with a test-only hook, `beachcomber.DocsUseEngine` in `export_docs_test.go`; the Mount example moved to `modules/lighthouse/centrifugo/example_test.go` as `ExampleDriver_Routes`. - **Commit:** 44bd144 **5. [Rule 3 - Blocking] Sub-packages matched by the verify patterns had no Example** - **Found during:** Task 3 - **Issue:** `./modules/lighthouse/...` and `./modules/beachcomber/...` include `centrifugo` and `typesense`, which reported "no tests to run" for `^(Example|TestDocs)`, a listed failure condition. - **Fix:** Added `ExampleProxyHandler` and `ExampleDriver_Routes` (centrifugo) and `ExampleEngine_SearchIDs` (typesense), used on the Realtime and Search pages. - **Files modified:** modules/lighthouse/centrifugo/example_test.go, modules/beachcomber/typesense/example_test.go (beyond the plan's file list) - **Commit:** 44bd144 ### Other additions beyond the file list - `modules/festival/example_test.go` gained `ExampleBus_Collect` and `ExampleBus_UntilHandled` for the Events page (the plan allowed adding a festival region). - `modules/cabana/example_controller_test.go` and `modules/cabana/testdata/docs/` hold the controller, plugin and YAML the Backend pages show as whole-file and `yaml src=` copies. `modules/tide/testdata/docs/posts-spec.yaml` is the spec the Parity testing page shows. - `TestDocsDeclarations` tests in lagoon, surf, cabana and beachcomber call the declarations that only GORM or the framework would call, so their `#Ident` fences pass the run check. **Total deviations:** 5 auto-fixed (2 blocking, 3 accuracy or bug). **Impact:** no module code changed; three module gaps are logged as todos. ## Issues Encountered - A debugging detour: a `grep -v "^20"` output filter hid the `200 ...` lines of an Example, which looked like a hang. No code was affected. - `gofmt -l` still lists pre-existing files (`modules/compass/config_test.go`, `modules/compass/persist_test.go`, `modules/tide/flow_test.go`, `modules/tide/headers_test.go`, `modules/wristband/server.go`), untouched by this plan. ## Verification - `go vet ./...` is clean; `go test -short ./...` is green. - `go test ./... -count=1` with Docker: every package passed except the two registry-list tests broken by task 3's first draft; after the fix, `go test ./modules/lighthouse/... ./modules/beachcomber/... -count=1` is green, and every `TestDocs*` test reports PASS (not SKIP). - `TestDocsTree`, `TestDocsRequiredPages` (with the section-order assertion), `TestDocsAIOutputsInSync` and `TestDocsBuildRealTree` pass; `docs:build` writes 71 pages; `docs:build --check` reports no problems and `docs:sync` reports every snippet up to date. - `scripts/check-phase11.1.sh --docs` and `--forbidden`, `scripts/check-phase10.sh --hygiene` and `scripts/check-phase11.sh --hygiene` pass. - Acceptance greps: 8 Backend, 7 Database and 16 Services pages; 8 of the 8 listed example files contain `// Output:`; the OAuth page has 4 wristband `src=` fences and none into `server.go`, `stores.go` or `client_issue.go`; `STARTTLS`, `lagoon.AfterCommit`, `Snowboard`, `WithoutBroadcasting` and `SearchIDs` appear where required. ## Known Stubs None. ## Threat Flags None. The pages add no runtime surface; the harness exports are `_test.go` files. ## User Setup Required None. ## Next Phase Readiness Plans 11.1-05 and 11.1-06 can build on a complete sidebar in the D-08 order. The three lagoon todos are ready for a code phase. ## Self-Check: PASSED All created files exist. Commits 9d37d56, efb35a2, f7dfe68 and 44bd144 are in the log.