docs(11.1-04): complete framework content B plan

This commit is contained in:
Jakub Zych
2026-09-30 23:20:20 +02:00
parent 44bd1446f5
commit 4e83d06025
3 changed files with 310 additions and 10 deletions

View File

@@ -544,7 +544,7 @@ Plans:
4. Every Go example in a `docs/` page is compiled and run by `go test ./...` (Go code in ingested module READMEs is identifier-checked, not compiled — D-18). An identifier checker and an internal link/anchor checker also run there and fail on stale names or broken links.
5. A "Coming from WinterCMS" concept map and an `acme/blog` porting walkthrough exist, and the walkthrough's code is verified under criterion 4.
**Plans:** 3/6 plans executed
**Plans:** 4/6 plans executed
Plans:
**Wave 1**
@@ -557,7 +557,7 @@ Plans:
- [x] 11.1-03-PLAN.md — Content A: Setup (incl. Coming from WinterCMS), Architecture, Plugins, Console, module Examples
**Wave 4** *(blocked on Wave 3 completion)*
- [ ] 11.1-04-PLAN.md — Content B: Database, Backend, Services (jobs, realtime, push, search, parity, transactions), Frontend and AJAX (not provided), module Examples
- [x] 11.1-04-PLAN.md — Content B: Database, Backend, Services (jobs, realtime, push, search, parity, transactions), Frontend and AJAX (not provided), module Examples
**Wave 5** *(blocked on Wave 4 completion)*
- [ ] 11.1-05-PLAN.md — acme/blog porting walkthrough under docs/examples/blog with Docker and scaffold-layout tests
@@ -674,7 +674,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 →
| 9. Backend admin authentication and schema pipeline | 12/12 | In Progress| |
| 10. Admin Vue SPA | 5/5 | Complete | 2026-09-27 |
| 11. Jobs, realtime and search infrastructure | 8/8 | In Progress| |
| 11.1. SummerCMS documentation for humans and AI agents | 3/6 | In Progress| |
| 11.1. SummerCMS documentation for humans and AI agents | 4/6 | In Progress| |
| 11.2. Ready to share: summercms.io website and newsletter plugin | 0/TBD | Not started | - |
| 12. Płytarium API — Collections and Albums | 0/TBD | Not started | - |
| 13. Płytarium API — wishlist, notifications, CSV, credentials, public routes | 0/TBD | Not started | - |

View File

@@ -4,16 +4,16 @@ milestone: v1.0
current_phase: "11.1"
current_phase_name: SummerCMS documentation for humans and AI agents (INSERTED)
status: executing
stopped_at: Completed 11.1-03-PLAN.md
last_updated: "2026-09-30T20:18:25.664Z"
stopped_at: Completed 11.1-04-PLAN.md
last_updated: "2026-09-30T21:20:10.781Z"
last_activity: 2026-09-30
last_activity_desc: Phase 11.1 execution started
state_head: 3131d7f3e1bf26765fd71fbbb04201d5d5d6c115
state_head: 44bd1446f572c92b31e16f7f6153349614af2c12
progress:
total_phases: 19
completed_phases: 9
total_plans: 92
completed_plans: 89
completed_plans: 90
milestone_name: milestone
---
@@ -29,7 +29,7 @@ See: .planning/PROJECT.md (updated 2026-09-16)
## Current Position
Phase: 11.1 (SummerCMS documentation for humans and AI agents (INSERTED)) — EXECUTING
Plan: 4 of 6
Plan: 5 of 6
Status: Ready to execute
Last activity: 2026-09-30 — Phase 11.1 execution started
@@ -144,6 +144,7 @@ Progress: [██████░░░░] 60%
| Phase 11.1 P01 | 15min | 3 tasks | 18 files |
| Phase 11.1 P02 | 30min | 3 tasks | 53 files |
| Phase 11.1 P03 | 15min | 3 tasks | 28 files |
| Phase 11.1 P04 | 47min | 3 tasks | 68 files |
## Accumulated Context
@@ -397,6 +398,9 @@ Recent decisions affecting current work:
- [Phase 11.1]: 11.1-03: the Plugin.php-in-Go section shows modules/party/example_plugin_test.go as a whole-file src= fence because Go methods cannot live in an Example body
- [Phase 11.1]: 11.1-03: docs pages never carry hand-written Go fences; where no runnable Example fits, the page uses prose
- [Phase 11.1]: 11.1-03: bonfire has no duplicate command name check; documented as is and logged as todo bonfire-duplicate-command-names
- [Phase 11.1]: 11.1-04: The transactions page follows lagoon at HEAD: nested lagoon.Transaction needs the outer tx, AfterCommit in a plain GORM transaction warns and skips
- [Phase 11.1]: 11.1-04: Docs examples never register global drivers or engines; toy engines use test-only hooks and driver examples live in the driver package
- [Phase 11.1]: 11.1-04: Database-bound docs snippets run as TestDocs* regions on each module's Postgres harness through export_docs_test.go wrappers
### Pending Todos
@@ -427,6 +431,6 @@ Items acknowledged and carried forward from previous milestone close:
## Session Continuity
Last session: 2026-09-30T20:18:25.015Z
Stopped at: Completed 11.1-03-PLAN.md
Last session: 2026-09-30T21:20:10.227Z
Stopped at: Completed 11.1-04-PLAN.md
Resume file: None

View File

@@ -0,0 +1,296 @@
---
phase: 11.1-summercms-documentation-for-humans-and-ai-agents
plan: 04
subsystem: docs
tags: [docs, wintercms, backend, database, services, examples, lagoon, conga, lighthouse, beachcomber, flare, tide, cabana]
requires:
- phase: 11.1-03
provides: "Setup, Architecture, Plugins and Console sections, the concept map, requiredPages and the example-file patterns"
- phase: 11-08
provides: "lagoon.Transaction and lagoon.AfterCommit gates (nested calls bound to the parent handle, foreign transactions fail closed)"
provides:
- "docs/backend: admin-controllers, forms, lists-and-filters, relation-manager, users-and-permissions, settings, partials-and-widgets, admin-spa"
- "docs/database: models, migrations, queries-and-pagination, relations, casts-and-validation, attachments, transactions"
- "docs/services: configuration, events, routing, rate-limiting, authentication, oauth-server, mail, localization, storage, outbound-http, jobs, realtime, push, search, parity-testing, frontend-and-ajax"
- "site.yaml sections in the D-08 order setup, architecture, plugins, backend, database, services, console, api, asserted by TestDocsRequiredPages"
- "Runnable Examples in 19 packages and database-backed TestDocs* regions in conga, lagoon, lighthouse and beachcomber through test-only harness exports"
- "Concept map rows linked to their guide pages; not-provided rows linked to the Frontend and AJAX page"
affects: [11.1-05, 11.1-06]
estimate_ref: "tokens 150000, tasks 3, confidence low"
actuals:
tokens: 71500
tasks: 3
commits: 4
plan_head_before: 69dd5764bb6157e53ef0559879df6a68c85a26dd
plan_head_after: 44bd1446f572c92b31e16f7f6153349614af2c12
tech-stack:
added: []
patterns:
- "Database-bound snippets are docs:start regions in a helper that a TestDocs* test calls with a database from an exported test-only harness wrapper (export_docs_test.go: conga.DocsApp, lagoon.DocsDB, lighthouse.DocsEnv, beachcomber.DocsApp)"
- "A declaration shown by #Ident must be reachable from a Test or Example; TestDocsDeclarations tests call the methods GORM or the framework would call (hooks, Migrations, Permissions, Searchable methods)"
- "Docs examples never register a global driver or engine: the registered-name lists are asserted by package tests, so a toy engine is installed with a test-only hook (beachcomber.DocsUseEngine) and driver-specific examples live in the driver's package"
- "YAML shown on Backend pages is a yaml src= copy of modules/cabana/testdata/docs fixtures that an Example compiles"
key-files:
created:
- docs/backend/admin-controllers.md
- docs/backend/forms.md
- docs/backend/lists-and-filters.md
- docs/backend/relation-manager.md
- docs/backend/users-and-permissions.md
- docs/backend/settings.md
- docs/backend/partials-and-widgets.md
- docs/backend/admin-spa.md
- docs/database/models.md
- docs/database/migrations.md
- docs/database/queries-and-pagination.md
- docs/database/relations.md
- docs/database/casts-and-validation.md
- docs/database/attachments.md
- docs/database/transactions.md
- docs/services/configuration.md
- docs/services/events.md
- docs/services/routing.md
- docs/services/rate-limiting.md
- docs/services/authentication.md
- docs/services/oauth-server.md
- docs/services/mail.md
- docs/services/localization.md
- docs/services/storage.md
- docs/services/outbound-http.md
- docs/services/jobs.md
- docs/services/realtime.md
- docs/services/push.md
- docs/services/search.md
- docs/services/parity-testing.md
- docs/services/frontend-and-ajax.md
- modules/conga/example_test.go
- modules/conga/export_docs_test.go
- modules/lagoon/example_test.go
- modules/lagoon/export_docs_test.go
- modules/lagoon/attach/example_test.go
- modules/compass/example_test.go
- modules/surf/example_test.go
- modules/wire/example_test.go
- modules/bouncer/example_test.go
- modules/wristband/example_test.go
- modules/postcard/example_test.go
- modules/phrasebook/example_test.go
- modules/cabana/example_test.go
- modules/cabana/example_controller_test.go
- modules/cabana/testdata/docs/
- modules/fetchguard/example_test.go
- modules/lighthouse/example_test.go
- modules/lighthouse/export_docs_test.go
- modules/lighthouse/centrifugo/example_test.go
- modules/flare/example_test.go
- modules/beachcomber/example_test.go
- modules/beachcomber/export_docs_test.go
- modules/beachcomber/typesense/example_test.go
- modules/tide/example_test.go
- modules/tide/testdata/docs/posts-spec.yaml
- .planning/todos/pending/lagoon-validate-min-message.md
- .planning/todos/pending/lagoon-readme-after-commit-callback-order.md
modified:
- docs/site.yaml
- docs/index.md
- docs/setup/coming-from-wintercms.md
- modules/festival/example_test.go
- cmd/summer/docs_test.go
key-decisions:
- "The transactions page follows lagoon at HEAD (after 9526b6b and 93c735b), not only 11-08-SUMMARY: a nested lagoon.Transaction must get the outer tx and errors on a root handle; AfterCommit inside a plain GORM transaction warns and skips"
- "The OnDatabase example registers its callback After(gorm:create).Before(gorm:commit_or_rollback_transaction): the lagoon README's After(gorm:after_create) form sorts after lagoon:after_commit and silently drops the work (reproduced by TestDocsOnDatabase, logged as a todo)"
- "Pivot rows are read in order through an explicit join; the page warns that ordering a many2many Preload by a pivot column fails in GORM"
- "Validation pages describe lagoon as it is, including that a numeric min failure is reported with the max message; the fix is a logged todo, not a module change"
- "Backend YAML comes from new testdata/docs fixtures compiled by ExampleCompileList, ExampleCompileForm and ExampleActivate rather than hand-written blocks"
- "wristband snippets come only from modules/wristband/example_test.go and set their own Issuer and Resource; the default resource URL is never quoted"
- "Docs examples do not register global drivers or engines, because lighthouse and beachcomber tests assert the registered-name lists"
patterns-established:
- "export_docs_test.go per database-backed module exposes its harness to the external example package"
- "TestDocsRequiredPages checks the site.yaml section order against sectionOrder"
requirements-completed: [DOCS-01, DOCS-04, DOCS-06]
coverage:
- id: D1
description: "Backend, Database and Services sections are in the sidebar in the D-08 order and all 31 new pages build as .html and .md"
requirement: DOCS-01
verification:
- kind: unit
ref: "cmd/summer/docs_test.go#TestDocsRequiredPages"
status: pass
- kind: unit
ref: "cmd/summer/docs_test.go#TestDocsAIOutputsInSync"
status: pass
- kind: unit
ref: "cmd/summer/docs_test.go#TestDocsBuildRealTree"
status: pass
human_judgment: false
- id: D2
description: "Every Go fence on the new pages is a src= copy of an Example with // Output: or a docs region that a TestDocs* test runs against the module's Postgres harness"
requirement: DOCS-04
verification:
- kind: integration
ref: "modules/conga/example_test.go#TestDocsDispatch"
status: pass
- kind: integration
ref: "modules/lagoon/example_test.go#TestDocsTransactions"
status: pass
- kind: integration
ref: "modules/lighthouse/example_test.go#TestDocsBroadcast"
status: pass
- kind: integration
ref: "modules/beachcomber/example_test.go#TestDocsSearch"
status: pass
- kind: other
ref: "go run ./cmd/summer docs:build --check"
status: pass
human_judgment: false
- id: D3
description: "The concept map links every guide page and the not-provided rows link the Frontend and AJAX page, which states that CMS pages, themes, components, the AJAX framework and Snowboard are not provided"
requirement: DOCS-06
verification:
- kind: unit
ref: "cmd/summer/docs_test.go#TestDocsTree"
status: pass
- kind: other
ref: "grep -c 'frontend-and-ajax.md' docs/setup/coming-from-wintercms.md (4)"
status: pass
human_judgment: false
- id: D4
description: "No page or output names a consuming application; phase 10 and 11 hygiene gates pass over the new example files"
verification:
- kind: other
ref: "scripts/check-phase11.1.sh --docs and --forbidden; scripts/check-phase10.sh --hygiene; scripts/check-phase11.sh --hygiene"
status: pass
human_judgment: false
- id: D5
description: "Pages are accurate to the modules, state the secure defaults (mandatory STARTTLS, allowlisted https push hosts, SQL re-gate of SearchIDs, secret-checked subscribe proxy) and read well for a WinterCMS developer"
verification: []
human_judgment: true
rationale: "Backstop truth in the plan: accuracy against the READMEs and usefulness are a reader's judgment"
duration: 47min
completed: 2026-09-30
status: complete
---
# Phase 11.1 Plan 04: Framework content B (Backend, Database, Services) Summary
**Thirty-one new guide pages cover the admin backend, the GORM data layer with lagoon transactions and after-commit work, and every framework service from configuration to Web Push, search and parity testing, plus a Frontend and AJAX (not provided) page. Every Go block is an Example or a docs region that `go test` runs, the database ones against each module's Postgres container.**
## Performance
- **Duration:** about 47 min
- **Started:** 2026-09-30T20:31:46Z
- **Completed:** 2026-09-30T21:19Z
- **Tasks:** 3
- **Files modified:** 68
## Accomplishments
- **Services section and Jobs tracer:** `docs/services/jobs.md` shows a typed job, its registration through `pact.HasJobs`, dispatch inside the caller's transaction, the `summer_jobs` statuses, progress and cancellation and the worker commands. `TestDocsDispatch` starts a real worker on the conga harness and waits for the dispatched job to complete its row.
- **Database section:** models (Fill, Hidden, hooks, the models-leaf rule), migrations (gormigrate sets, history tables, rollback), queries and pagination (allow-listed `OrderBy`, `Paginate`), relations (pivot models, soft-delete cascades), casts and validation (Jsonable, Encrypted with key rotation, Laravel rule strings), attachments (system_files, thumbnails, two-phase delete) and transactions (AfterCommit rules table, savepoints, the root-handle refusal, `OnDatabase` callbacks). Six `TestDocs*` tests run the regions on the lagoon harness.
- **Core Services:** configuration layers, events (Fire, Collect, UntilHandled with new festival Examples), routing with an auth group built from a bouncer guard, rate limiting and trusted proxies, authentication (tokens, refresh, blacklist, guards, bcrypt), the OAuth server (no default resource quoted), mail (STARTTLS mandatory by default, `starttls`/`none` marked development-only) and localization (CLDR plurals with post/posty/postów).
- **Backend section:** admin controllers, forms, lists and filters, relation manager, users and permissions, settings, partials and widgets and the admin SPA. YAML blocks are copies of new `modules/cabana/testdata/docs` fixtures that `ExampleCompileList`, `ExampleCompileForm` and `ExampleActivate` compile.
- **Phase 11 services as shipped:** realtime (authorizers on every subscribe, `lighthouse.Mount` surfaces, broadcasts enqueued in the write transaction and published after commit, `WithoutBroadcasting` plus `Emit`, the Centrifugo proxy answering 200 with a constant-time secret check), Web Push (https only, allowlisted hosts, no redirects, VAPID key handling), search (after-commit sync, kill-switch gate, stale candidates re-checked in SQL, Typesense requests) and parity testing (record, replay, masked diffs, broadcast goldens).
- **Frontend and AJAX (not provided):** states the headless model and what is not provided, and routes readers to routing, authentication and realtime. The concept map now links a guide page on every row that has one.
- `TestDocsRequiredPages` covers 49 pages and asserts the section order `setup, architecture, plugins, backend, database, services, console, api`.
## Task Commits
1. **Task 1: Tracer, the Services section with a verified Jobs page:** `9d37d56` (feat)
2. **Task 2: Database section and the core Services pages:** `efb35a2` (feat)
3. **Todos for the lagoon gaps found in Task 2:** `f7dfe68` (docs)
4. **Task 3: Backend section, remaining Services pages and concept-map links:** `44bd144` (feat)
**Plan metadata:** the docs(11.1-04) commit that adds this file
## Files Created/Modified
See `key-files` in the frontmatter.
## Decisions Made
See `key-decisions` in the frontmatter.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 1 - Bug] The lagoon README's after-commit callback pattern drops its work**
- **Found during:** Task 2
- **Issue:** A create callback registered `After("gorm:after_create")` that calls `lagoon.AfterCommit` is sorted after `lagoon:after_commit`, so its buffered work never runs. `TestDocsOnDatabase` reproduced it.
- **Fix:** The docs region registers `After("gorm:create").Before("gorm:commit_or_rollback_transaction")` and the Transactions page warns about the ordering. The README (read-only for this plan) is logged in `.planning/todos/pending/lagoon-readme-after-commit-callback-order.md`.
- **Commit:** efb35a2, f7dfe68
**2. [Rule 1 - Accuracy] Pivot ordering through Preload fails**
- **Found during:** Task 2
- **Issue:** Ordering a many2many `Preload` by a pivot column fails with "missing FROM-clause entry", although the lagoon comment and README describe it.
- **Fix:** The Relations region reads through an explicit join; the page carries a warning; the README claim is in the same todo.
- **Commit:** efb35a2
**3. [Rule 1 - Accuracy] Numeric min failures use the max message**
- **Found during:** Task 2
- **Issue:** `lagoon.Validate` answers a `min` failure on an integer field with "may not be greater than ." (empty limit).
- **Fix:** The example uses `max`; the page has a NOTE describing the behaviour; the fix is logged in `.planning/todos/pending/lagoon-validate-min-message.md`. No module change.
- **Commit:** efb35a2, f7dfe68
**4. [Rule 3 - Blocking] Docs examples changed the registered driver and engine lists**
- **Found during:** Task 3 (full `go test ./...` run)
- **Issue:** Registering an `acme-memory` search engine from an example's `init`, and importing the centrifugo driver into the lighthouse example package, changed the registered-name lists that `TestServiceSetup` (beachcomber, read-only `sync_test.go`) and `TestFromSelectsDriver` (lighthouse) assert.
- **Fix:** The toy engine is installed with a test-only hook, `beachcomber.DocsUseEngine` in `export_docs_test.go`; the Mount example moved to `modules/lighthouse/centrifugo/example_test.go` as `ExampleDriver_Routes`.
- **Commit:** 44bd144
**5. [Rule 3 - Blocking] Sub-packages matched by the verify patterns had no Example**
- **Found during:** Task 3
- **Issue:** `./modules/lighthouse/...` and `./modules/beachcomber/...` include `centrifugo` and `typesense`, which reported "no tests to run" for `^(Example|TestDocs)`, a listed failure condition.
- **Fix:** Added `ExampleProxyHandler` and `ExampleDriver_Routes` (centrifugo) and `ExampleEngine_SearchIDs` (typesense), used on the Realtime and Search pages.
- **Files modified:** modules/lighthouse/centrifugo/example_test.go, modules/beachcomber/typesense/example_test.go (beyond the plan's file list)
- **Commit:** 44bd144
### Other additions beyond the file list
- `modules/festival/example_test.go` gained `ExampleBus_Collect` and `ExampleBus_UntilHandled` for the Events page (the plan allowed adding a festival region).
- `modules/cabana/example_controller_test.go` and `modules/cabana/testdata/docs/` hold the controller, plugin and YAML the Backend pages show as whole-file and `yaml src=` copies. `modules/tide/testdata/docs/posts-spec.yaml` is the spec the Parity testing page shows.
- `TestDocsDeclarations` tests in lagoon, surf, cabana and beachcomber call the declarations that only GORM or the framework would call, so their `#Ident` fences pass the run check.
**Total deviations:** 5 auto-fixed (2 blocking, 3 accuracy or bug). **Impact:** no module code changed; three module gaps are logged as todos.
## Issues Encountered
- A debugging detour: a `grep -v "^20"` output filter hid the `200 ...` lines of an Example, which looked like a hang. No code was affected.
- `gofmt -l` still lists pre-existing files (`modules/compass/config_test.go`, `modules/compass/persist_test.go`, `modules/tide/flow_test.go`, `modules/tide/headers_test.go`, `modules/wristband/server.go`), untouched by this plan.
## Verification
- `go vet ./...` is clean; `go test -short ./...` is green.
- `go test ./... -count=1` with Docker: every package passed except the two registry-list tests broken by task 3's first draft; after the fix, `go test ./modules/lighthouse/... ./modules/beachcomber/... -count=1` is green, and every `TestDocs*` test reports PASS (not SKIP).
- `TestDocsTree`, `TestDocsRequiredPages` (with the section-order assertion), `TestDocsAIOutputsInSync` and `TestDocsBuildRealTree` pass; `docs:build` writes 71 pages; `docs:build --check` reports no problems and `docs:sync` reports every snippet up to date.
- `scripts/check-phase11.1.sh --docs` and `--forbidden`, `scripts/check-phase10.sh --hygiene` and `scripts/check-phase11.sh --hygiene` pass.
- Acceptance greps: 8 Backend, 7 Database and 16 Services pages; 8 of the 8 listed example files contain `// Output:`; the OAuth page has 4 wristband `src=` fences and none into `server.go`, `stores.go` or `client_issue.go`; `STARTTLS`, `lagoon.AfterCommit`, `Snowboard`, `WithoutBroadcasting` and `SearchIDs` appear where required.
## Known Stubs
None.
## Threat Flags
None. The pages add no runtime surface; the harness exports are `_test.go` files.
## User Setup Required
None.
## Next Phase Readiness
Plans 11.1-05 and 11.1-06 can build on a complete sidebar in the D-08 order. The three lagoon todos are ready for a code phase.
## Self-Check: PASSED
All created files exist. Commits 9d37d56, efb35a2, f7dfe68 and 44bd144 are in the log.