From 4e83d06025af7993233caf8fd7bf16ea7a2fcc83 Mon Sep 17 00:00:00 2001 From: Jakub Zych Date: Wed, 30 Sep 2026 23:20:20 +0200 Subject: [PATCH] docs(11.1-04): complete framework content B plan --- .planning/ROADMAP.md | 6 +- .planning/STATE.md | 18 +- .../11.1-04-SUMMARY.md | 296 ++++++++++++++++++ 3 files changed, 310 insertions(+), 10 deletions(-) create mode 100644 .planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-04-SUMMARY.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index c0b5c76..efcf90d 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -544,7 +544,7 @@ Plans: 4. Every Go example in a `docs/` page is compiled and run by `go test ./...` (Go code in ingested module READMEs is identifier-checked, not compiled — D-18). An identifier checker and an internal link/anchor checker also run there and fail on stale names or broken links. 5. A "Coming from WinterCMS" concept map and an `acme/blog` porting walkthrough exist, and the walkthrough's code is verified under criterion 4. -**Plans:** 3/6 plans executed +**Plans:** 4/6 plans executed Plans: **Wave 1** @@ -557,7 +557,7 @@ Plans: - [x] 11.1-03-PLAN.md — Content A: Setup (incl. Coming from WinterCMS), Architecture, Plugins, Console, module Examples **Wave 4** *(blocked on Wave 3 completion)* -- [ ] 11.1-04-PLAN.md — Content B: Database, Backend, Services (jobs, realtime, push, search, parity, transactions), Frontend and AJAX (not provided), module Examples +- [x] 11.1-04-PLAN.md — Content B: Database, Backend, Services (jobs, realtime, push, search, parity, transactions), Frontend and AJAX (not provided), module Examples **Wave 5** *(blocked on Wave 4 completion)* - [ ] 11.1-05-PLAN.md — acme/blog porting walkthrough under docs/examples/blog with Docker and scaffold-layout tests @@ -674,7 +674,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → | 9. Backend admin authentication and schema pipeline | 12/12 | In Progress| | | 10. Admin Vue SPA | 5/5 | Complete | 2026-09-27 | | 11. Jobs, realtime and search infrastructure | 8/8 | In Progress| | -| 11.1. SummerCMS documentation for humans and AI agents | 3/6 | In Progress| | +| 11.1. SummerCMS documentation for humans and AI agents | 4/6 | In Progress| | | 11.2. Ready to share: summercms.io website and newsletter plugin | 0/TBD | Not started | - | | 12. Płytarium API — Collections and Albums | 0/TBD | Not started | - | | 13. Płytarium API — wishlist, notifications, CSV, credentials, public routes | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 51d440e..43486db 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -4,16 +4,16 @@ milestone: v1.0 current_phase: "11.1" current_phase_name: SummerCMS documentation for humans and AI agents (INSERTED) status: executing -stopped_at: Completed 11.1-03-PLAN.md -last_updated: "2026-09-30T20:18:25.664Z" +stopped_at: Completed 11.1-04-PLAN.md +last_updated: "2026-09-30T21:20:10.781Z" last_activity: 2026-09-30 last_activity_desc: Phase 11.1 execution started -state_head: 3131d7f3e1bf26765fd71fbbb04201d5d5d6c115 +state_head: 44bd1446f572c92b31e16f7f6153349614af2c12 progress: total_phases: 19 completed_phases: 9 total_plans: 92 - completed_plans: 89 + completed_plans: 90 milestone_name: milestone --- @@ -29,7 +29,7 @@ See: .planning/PROJECT.md (updated 2026-09-16) ## Current Position Phase: 11.1 (SummerCMS documentation for humans and AI agents (INSERTED)) — EXECUTING -Plan: 4 of 6 +Plan: 5 of 6 Status: Ready to execute Last activity: 2026-09-30 — Phase 11.1 execution started @@ -144,6 +144,7 @@ Progress: [██████░░░░] 60% | Phase 11.1 P01 | 15min | 3 tasks | 18 files | | Phase 11.1 P02 | 30min | 3 tasks | 53 files | | Phase 11.1 P03 | 15min | 3 tasks | 28 files | +| Phase 11.1 P04 | 47min | 3 tasks | 68 files | ## Accumulated Context @@ -397,6 +398,9 @@ Recent decisions affecting current work: - [Phase 11.1]: 11.1-03: the Plugin.php-in-Go section shows modules/party/example_plugin_test.go as a whole-file src= fence because Go methods cannot live in an Example body - [Phase 11.1]: 11.1-03: docs pages never carry hand-written Go fences; where no runnable Example fits, the page uses prose - [Phase 11.1]: 11.1-03: bonfire has no duplicate command name check; documented as is and logged as todo bonfire-duplicate-command-names +- [Phase 11.1]: 11.1-04: The transactions page follows lagoon at HEAD: nested lagoon.Transaction needs the outer tx, AfterCommit in a plain GORM transaction warns and skips +- [Phase 11.1]: 11.1-04: Docs examples never register global drivers or engines; toy engines use test-only hooks and driver examples live in the driver package +- [Phase 11.1]: 11.1-04: Database-bound docs snippets run as TestDocs* regions on each module's Postgres harness through export_docs_test.go wrappers ### Pending Todos @@ -427,6 +431,6 @@ Items acknowledged and carried forward from previous milestone close: ## Session Continuity -Last session: 2026-09-30T20:18:25.015Z -Stopped at: Completed 11.1-03-PLAN.md +Last session: 2026-09-30T21:20:10.227Z +Stopped at: Completed 11.1-04-PLAN.md Resume file: None diff --git a/.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-04-SUMMARY.md b/.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-04-SUMMARY.md new file mode 100644 index 0000000..6aeac15 --- /dev/null +++ b/.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-04-SUMMARY.md @@ -0,0 +1,296 @@ +--- +phase: 11.1-summercms-documentation-for-humans-and-ai-agents +plan: 04 +subsystem: docs +tags: [docs, wintercms, backend, database, services, examples, lagoon, conga, lighthouse, beachcomber, flare, tide, cabana] + +requires: + - phase: 11.1-03 + provides: "Setup, Architecture, Plugins and Console sections, the concept map, requiredPages and the example-file patterns" + - phase: 11-08 + provides: "lagoon.Transaction and lagoon.AfterCommit gates (nested calls bound to the parent handle, foreign transactions fail closed)" +provides: + - "docs/backend: admin-controllers, forms, lists-and-filters, relation-manager, users-and-permissions, settings, partials-and-widgets, admin-spa" + - "docs/database: models, migrations, queries-and-pagination, relations, casts-and-validation, attachments, transactions" + - "docs/services: configuration, events, routing, rate-limiting, authentication, oauth-server, mail, localization, storage, outbound-http, jobs, realtime, push, search, parity-testing, frontend-and-ajax" + - "site.yaml sections in the D-08 order setup, architecture, plugins, backend, database, services, console, api, asserted by TestDocsRequiredPages" + - "Runnable Examples in 19 packages and database-backed TestDocs* regions in conga, lagoon, lighthouse and beachcomber through test-only harness exports" + - "Concept map rows linked to their guide pages; not-provided rows linked to the Frontend and AJAX page" +affects: [11.1-05, 11.1-06] + +estimate_ref: "tokens 150000, tasks 3, confidence low" +actuals: + tokens: 71500 + tasks: 3 + commits: 4 +plan_head_before: 69dd5764bb6157e53ef0559879df6a68c85a26dd +plan_head_after: 44bd1446f572c92b31e16f7f6153349614af2c12 + +tech-stack: + added: [] + patterns: + - "Database-bound snippets are docs:start regions in a helper that a TestDocs* test calls with a database from an exported test-only harness wrapper (export_docs_test.go: conga.DocsApp, lagoon.DocsDB, lighthouse.DocsEnv, beachcomber.DocsApp)" + - "A declaration shown by #Ident must be reachable from a Test or Example; TestDocsDeclarations tests call the methods GORM or the framework would call (hooks, Migrations, Permissions, Searchable methods)" + - "Docs examples never register a global driver or engine: the registered-name lists are asserted by package tests, so a toy engine is installed with a test-only hook (beachcomber.DocsUseEngine) and driver-specific examples live in the driver's package" + - "YAML shown on Backend pages is a yaml src= copy of modules/cabana/testdata/docs fixtures that an Example compiles" + +key-files: + created: + - docs/backend/admin-controllers.md + - docs/backend/forms.md + - docs/backend/lists-and-filters.md + - docs/backend/relation-manager.md + - docs/backend/users-and-permissions.md + - docs/backend/settings.md + - docs/backend/partials-and-widgets.md + - docs/backend/admin-spa.md + - docs/database/models.md + - docs/database/migrations.md + - docs/database/queries-and-pagination.md + - docs/database/relations.md + - docs/database/casts-and-validation.md + - docs/database/attachments.md + - docs/database/transactions.md + - docs/services/configuration.md + - docs/services/events.md + - docs/services/routing.md + - docs/services/rate-limiting.md + - docs/services/authentication.md + - docs/services/oauth-server.md + - docs/services/mail.md + - docs/services/localization.md + - docs/services/storage.md + - docs/services/outbound-http.md + - docs/services/jobs.md + - docs/services/realtime.md + - docs/services/push.md + - docs/services/search.md + - docs/services/parity-testing.md + - docs/services/frontend-and-ajax.md + - modules/conga/example_test.go + - modules/conga/export_docs_test.go + - modules/lagoon/example_test.go + - modules/lagoon/export_docs_test.go + - modules/lagoon/attach/example_test.go + - modules/compass/example_test.go + - modules/surf/example_test.go + - modules/wire/example_test.go + - modules/bouncer/example_test.go + - modules/wristband/example_test.go + - modules/postcard/example_test.go + - modules/phrasebook/example_test.go + - modules/cabana/example_test.go + - modules/cabana/example_controller_test.go + - modules/cabana/testdata/docs/ + - modules/fetchguard/example_test.go + - modules/lighthouse/example_test.go + - modules/lighthouse/export_docs_test.go + - modules/lighthouse/centrifugo/example_test.go + - modules/flare/example_test.go + - modules/beachcomber/example_test.go + - modules/beachcomber/export_docs_test.go + - modules/beachcomber/typesense/example_test.go + - modules/tide/example_test.go + - modules/tide/testdata/docs/posts-spec.yaml + - .planning/todos/pending/lagoon-validate-min-message.md + - .planning/todos/pending/lagoon-readme-after-commit-callback-order.md + modified: + - docs/site.yaml + - docs/index.md + - docs/setup/coming-from-wintercms.md + - modules/festival/example_test.go + - cmd/summer/docs_test.go + +key-decisions: + - "The transactions page follows lagoon at HEAD (after 9526b6b and 93c735b), not only 11-08-SUMMARY: a nested lagoon.Transaction must get the outer tx and errors on a root handle; AfterCommit inside a plain GORM transaction warns and skips" + - "The OnDatabase example registers its callback After(gorm:create).Before(gorm:commit_or_rollback_transaction): the lagoon README's After(gorm:after_create) form sorts after lagoon:after_commit and silently drops the work (reproduced by TestDocsOnDatabase, logged as a todo)" + - "Pivot rows are read in order through an explicit join; the page warns that ordering a many2many Preload by a pivot column fails in GORM" + - "Validation pages describe lagoon as it is, including that a numeric min failure is reported with the max message; the fix is a logged todo, not a module change" + - "Backend YAML comes from new testdata/docs fixtures compiled by ExampleCompileList, ExampleCompileForm and ExampleActivate rather than hand-written blocks" + - "wristband snippets come only from modules/wristband/example_test.go and set their own Issuer and Resource; the default resource URL is never quoted" + - "Docs examples do not register global drivers or engines, because lighthouse and beachcomber tests assert the registered-name lists" + +patterns-established: + - "export_docs_test.go per database-backed module exposes its harness to the external example package" + - "TestDocsRequiredPages checks the site.yaml section order against sectionOrder" + +requirements-completed: [DOCS-01, DOCS-04, DOCS-06] + +coverage: + - id: D1 + description: "Backend, Database and Services sections are in the sidebar in the D-08 order and all 31 new pages build as .html and .md" + requirement: DOCS-01 + verification: + - kind: unit + ref: "cmd/summer/docs_test.go#TestDocsRequiredPages" + status: pass + - kind: unit + ref: "cmd/summer/docs_test.go#TestDocsAIOutputsInSync" + status: pass + - kind: unit + ref: "cmd/summer/docs_test.go#TestDocsBuildRealTree" + status: pass + human_judgment: false + - id: D2 + description: "Every Go fence on the new pages is a src= copy of an Example with // Output: or a docs region that a TestDocs* test runs against the module's Postgres harness" + requirement: DOCS-04 + verification: + - kind: integration + ref: "modules/conga/example_test.go#TestDocsDispatch" + status: pass + - kind: integration + ref: "modules/lagoon/example_test.go#TestDocsTransactions" + status: pass + - kind: integration + ref: "modules/lighthouse/example_test.go#TestDocsBroadcast" + status: pass + - kind: integration + ref: "modules/beachcomber/example_test.go#TestDocsSearch" + status: pass + - kind: other + ref: "go run ./cmd/summer docs:build --check" + status: pass + human_judgment: false + - id: D3 + description: "The concept map links every guide page and the not-provided rows link the Frontend and AJAX page, which states that CMS pages, themes, components, the AJAX framework and Snowboard are not provided" + requirement: DOCS-06 + verification: + - kind: unit + ref: "cmd/summer/docs_test.go#TestDocsTree" + status: pass + - kind: other + ref: "grep -c 'frontend-and-ajax.md' docs/setup/coming-from-wintercms.md (4)" + status: pass + human_judgment: false + - id: D4 + description: "No page or output names a consuming application; phase 10 and 11 hygiene gates pass over the new example files" + verification: + - kind: other + ref: "scripts/check-phase11.1.sh --docs and --forbidden; scripts/check-phase10.sh --hygiene; scripts/check-phase11.sh --hygiene" + status: pass + human_judgment: false + - id: D5 + description: "Pages are accurate to the modules, state the secure defaults (mandatory STARTTLS, allowlisted https push hosts, SQL re-gate of SearchIDs, secret-checked subscribe proxy) and read well for a WinterCMS developer" + verification: [] + human_judgment: true + rationale: "Backstop truth in the plan: accuracy against the READMEs and usefulness are a reader's judgment" + +duration: 47min +completed: 2026-09-30 +status: complete +--- + +# Phase 11.1 Plan 04: Framework content B (Backend, Database, Services) Summary + +**Thirty-one new guide pages cover the admin backend, the GORM data layer with lagoon transactions and after-commit work, and every framework service from configuration to Web Push, search and parity testing, plus a Frontend and AJAX (not provided) page. Every Go block is an Example or a docs region that `go test` runs, the database ones against each module's Postgres container.** + +## Performance + +- **Duration:** about 47 min +- **Started:** 2026-09-30T20:31:46Z +- **Completed:** 2026-09-30T21:19Z +- **Tasks:** 3 +- **Files modified:** 68 + +## Accomplishments + +- **Services section and Jobs tracer:** `docs/services/jobs.md` shows a typed job, its registration through `pact.HasJobs`, dispatch inside the caller's transaction, the `summer_jobs` statuses, progress and cancellation and the worker commands. `TestDocsDispatch` starts a real worker on the conga harness and waits for the dispatched job to complete its row. +- **Database section:** models (Fill, Hidden, hooks, the models-leaf rule), migrations (gormigrate sets, history tables, rollback), queries and pagination (allow-listed `OrderBy`, `Paginate`), relations (pivot models, soft-delete cascades), casts and validation (Jsonable, Encrypted with key rotation, Laravel rule strings), attachments (system_files, thumbnails, two-phase delete) and transactions (AfterCommit rules table, savepoints, the root-handle refusal, `OnDatabase` callbacks). Six `TestDocs*` tests run the regions on the lagoon harness. +- **Core Services:** configuration layers, events (Fire, Collect, UntilHandled with new festival Examples), routing with an auth group built from a bouncer guard, rate limiting and trusted proxies, authentication (tokens, refresh, blacklist, guards, bcrypt), the OAuth server (no default resource quoted), mail (STARTTLS mandatory by default, `starttls`/`none` marked development-only) and localization (CLDR plurals with post/posty/postów). +- **Backend section:** admin controllers, forms, lists and filters, relation manager, users and permissions, settings, partials and widgets and the admin SPA. YAML blocks are copies of new `modules/cabana/testdata/docs` fixtures that `ExampleCompileList`, `ExampleCompileForm` and `ExampleActivate` compile. +- **Phase 11 services as shipped:** realtime (authorizers on every subscribe, `lighthouse.Mount` surfaces, broadcasts enqueued in the write transaction and published after commit, `WithoutBroadcasting` plus `Emit`, the Centrifugo proxy answering 200 with a constant-time secret check), Web Push (https only, allowlisted hosts, no redirects, VAPID key handling), search (after-commit sync, kill-switch gate, stale candidates re-checked in SQL, Typesense requests) and parity testing (record, replay, masked diffs, broadcast goldens). +- **Frontend and AJAX (not provided):** states the headless model and what is not provided, and routes readers to routing, authentication and realtime. The concept map now links a guide page on every row that has one. +- `TestDocsRequiredPages` covers 49 pages and asserts the section order `setup, architecture, plugins, backend, database, services, console, api`. + +## Task Commits + +1. **Task 1: Tracer, the Services section with a verified Jobs page:** `9d37d56` (feat) +2. **Task 2: Database section and the core Services pages:** `efb35a2` (feat) +3. **Todos for the lagoon gaps found in Task 2:** `f7dfe68` (docs) +4. **Task 3: Backend section, remaining Services pages and concept-map links:** `44bd144` (feat) + +**Plan metadata:** the docs(11.1-04) commit that adds this file + +## Files Created/Modified + +See `key-files` in the frontmatter. + +## Decisions Made + +See `key-decisions` in the frontmatter. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] The lagoon README's after-commit callback pattern drops its work** +- **Found during:** Task 2 +- **Issue:** A create callback registered `After("gorm:after_create")` that calls `lagoon.AfterCommit` is sorted after `lagoon:after_commit`, so its buffered work never runs. `TestDocsOnDatabase` reproduced it. +- **Fix:** The docs region registers `After("gorm:create").Before("gorm:commit_or_rollback_transaction")` and the Transactions page warns about the ordering. The README (read-only for this plan) is logged in `.planning/todos/pending/lagoon-readme-after-commit-callback-order.md`. +- **Commit:** efb35a2, f7dfe68 + +**2. [Rule 1 - Accuracy] Pivot ordering through Preload fails** +- **Found during:** Task 2 +- **Issue:** Ordering a many2many `Preload` by a pivot column fails with "missing FROM-clause entry", although the lagoon comment and README describe it. +- **Fix:** The Relations region reads through an explicit join; the page carries a warning; the README claim is in the same todo. +- **Commit:** efb35a2 + +**3. [Rule 1 - Accuracy] Numeric min failures use the max message** +- **Found during:** Task 2 +- **Issue:** `lagoon.Validate` answers a `min` failure on an integer field with "may not be greater than ." (empty limit). +- **Fix:** The example uses `max`; the page has a NOTE describing the behaviour; the fix is logged in `.planning/todos/pending/lagoon-validate-min-message.md`. No module change. +- **Commit:** efb35a2, f7dfe68 + +**4. [Rule 3 - Blocking] Docs examples changed the registered driver and engine lists** +- **Found during:** Task 3 (full `go test ./...` run) +- **Issue:** Registering an `acme-memory` search engine from an example's `init`, and importing the centrifugo driver into the lighthouse example package, changed the registered-name lists that `TestServiceSetup` (beachcomber, read-only `sync_test.go`) and `TestFromSelectsDriver` (lighthouse) assert. +- **Fix:** The toy engine is installed with a test-only hook, `beachcomber.DocsUseEngine` in `export_docs_test.go`; the Mount example moved to `modules/lighthouse/centrifugo/example_test.go` as `ExampleDriver_Routes`. +- **Commit:** 44bd144 + +**5. [Rule 3 - Blocking] Sub-packages matched by the verify patterns had no Example** +- **Found during:** Task 3 +- **Issue:** `./modules/lighthouse/...` and `./modules/beachcomber/...` include `centrifugo` and `typesense`, which reported "no tests to run" for `^(Example|TestDocs)`, a listed failure condition. +- **Fix:** Added `ExampleProxyHandler` and `ExampleDriver_Routes` (centrifugo) and `ExampleEngine_SearchIDs` (typesense), used on the Realtime and Search pages. +- **Files modified:** modules/lighthouse/centrifugo/example_test.go, modules/beachcomber/typesense/example_test.go (beyond the plan's file list) +- **Commit:** 44bd144 + +### Other additions beyond the file list + +- `modules/festival/example_test.go` gained `ExampleBus_Collect` and `ExampleBus_UntilHandled` for the Events page (the plan allowed adding a festival region). +- `modules/cabana/example_controller_test.go` and `modules/cabana/testdata/docs/` hold the controller, plugin and YAML the Backend pages show as whole-file and `yaml src=` copies. `modules/tide/testdata/docs/posts-spec.yaml` is the spec the Parity testing page shows. +- `TestDocsDeclarations` tests in lagoon, surf, cabana and beachcomber call the declarations that only GORM or the framework would call, so their `#Ident` fences pass the run check. + +**Total deviations:** 5 auto-fixed (2 blocking, 3 accuracy or bug). **Impact:** no module code changed; three module gaps are logged as todos. + +## Issues Encountered + +- A debugging detour: a `grep -v "^20"` output filter hid the `200 ...` lines of an Example, which looked like a hang. No code was affected. +- `gofmt -l` still lists pre-existing files (`modules/compass/config_test.go`, `modules/compass/persist_test.go`, `modules/tide/flow_test.go`, `modules/tide/headers_test.go`, `modules/wristband/server.go`), untouched by this plan. + +## Verification + +- `go vet ./...` is clean; `go test -short ./...` is green. +- `go test ./... -count=1` with Docker: every package passed except the two registry-list tests broken by task 3's first draft; after the fix, `go test ./modules/lighthouse/... ./modules/beachcomber/... -count=1` is green, and every `TestDocs*` test reports PASS (not SKIP). +- `TestDocsTree`, `TestDocsRequiredPages` (with the section-order assertion), `TestDocsAIOutputsInSync` and `TestDocsBuildRealTree` pass; `docs:build` writes 71 pages; `docs:build --check` reports no problems and `docs:sync` reports every snippet up to date. +- `scripts/check-phase11.1.sh --docs` and `--forbidden`, `scripts/check-phase10.sh --hygiene` and `scripts/check-phase11.sh --hygiene` pass. +- Acceptance greps: 8 Backend, 7 Database and 16 Services pages; 8 of the 8 listed example files contain `// Output:`; the OAuth page has 4 wristband `src=` fences and none into `server.go`, `stores.go` or `client_issue.go`; `STARTTLS`, `lagoon.AfterCommit`, `Snowboard`, `WithoutBroadcasting` and `SearchIDs` appear where required. + +## Known Stubs + +None. + +## Threat Flags + +None. The pages add no runtime surface; the harness exports are `_test.go` files. + +## User Setup Required + +None. + +## Next Phase Readiness + +Plans 11.1-05 and 11.1-06 can build on a complete sidebar in the D-08 order. The three lagoon todos are ready for a code phase. + +## Self-Check: PASSED + +All created files exist. Commits 9d37d56, efb35a2, f7dfe68 and 44bd144 are in the log.