docs(12.2-02): complete the fileupload and datepicker admin API plan summary
This commit is contained in:
Jakub Zych
@@ -0,0 +1,249 @@
|
|||||||
|
---
|
||||||
|
phase: 12.2-admin-form-fields-date-file-upload-relation-editing-with-def
|
||||||
|
plan: 02
|
||||||
|
subsystem: admin
|
||||||
|
tags: [cabana, fileupload, datepicker, deferred-binding, attach, openapi, swag]
|
||||||
|
|
||||||
|
requires:
|
||||||
|
- phase: 12.2-01
|
||||||
|
provides: deferred_bindings store ops, attach.Store, attach.Relation/HasRelations, File.ThumbKey, lagoon.Date/TimeOfDay, Fill text fallback
|
||||||
|
provides:
|
||||||
|
- type: fileupload (D-08 keys, boot-bound to attach.Relation, maxFilesize vs http.body_limits.upload_bytes)
|
||||||
|
- X-Session-Key (cabana.SessionKeyHeader) and RecordInput.SessionKey
|
||||||
|
- seven file routes under .../{id}/files/{field} (list, upload, caption, remove, reorder, protected download, protected thumb)
|
||||||
|
- commitDeferred in CRUDService.save: binds, unbinds, attachOne replace, maxFiles/required recheck
|
||||||
|
- type: datepicker (D-20 keys, displayFormat, D-19 Go type check, server min/max)
|
||||||
|
- columns.yaml type: date and type: time; Scanner/Valuer structs are not relations
|
||||||
|
- swagger2openapi: formData -> multipart requestBody, file responses -> binary
|
||||||
|
affects: [12.2-03 relation child CRUD (reuses fileScope, sessionKeyFrom, commitDeferred), 12.2-04 admin SPA, 12.2-05 unit and security tests]
|
||||||
|
|
||||||
|
actuals:
|
||||||
|
tokens: 66000
|
||||||
|
tasks: 3
|
||||||
|
commits: 4
|
||||||
|
plan_head_before: edda803dc11e6c8238fd043c0185858d26f132ea
|
||||||
|
plan_head_after: 67d4c7ff13a7c0f82f233dbf21db59654ee3934f
|
||||||
|
|
||||||
|
tech-stack:
|
||||||
|
added: []
|
||||||
|
patterns:
|
||||||
|
- "File handlers take a resolved fileScope (field, owner or id 0, DeferredKey), so a child scope can reuse them"
|
||||||
|
- "Every file id is resolved by one parent-scoped query; a miss is 404, never 403"
|
||||||
|
- "Blob deletes of files removed in a save run through lagoon.AfterCommit"
|
||||||
|
- "Binary admin routes are documented with @Success 200 {file} file and checked by a raw conformance case"
|
||||||
|
|
||||||
|
key-files:
|
||||||
|
created:
|
||||||
|
- modules/cabana/field_file.go
|
||||||
|
- modules/cabana/field_date.go
|
||||||
|
- modules/cabana/deferred.go
|
||||||
|
- modules/cabana/fileupload_smoke_test.go
|
||||||
|
- modules/cabana/datepicker_smoke_test.go
|
||||||
|
modified:
|
||||||
|
- modules/cabana/form_schema.go
|
||||||
|
- modules/cabana/schema_types.go
|
||||||
|
- modules/cabana/crud.go
|
||||||
|
- modules/cabana/http.go
|
||||||
|
- modules/cabana/registry.go
|
||||||
|
- modules/cabana/contracts.go
|
||||||
|
- modules/cabana/settings.go
|
||||||
|
- modules/cabana/list_schema.go
|
||||||
|
- modules/cabana/auth.go
|
||||||
|
- modules/cabana/admin_openapi.go
|
||||||
|
- modules/cabana/security_coverage_test.go
|
||||||
|
- modules/cabana/openapi_conformance_test.go
|
||||||
|
- modules/cabana/phase10_csrf_test.go
|
||||||
|
- modules/cabana/phase10_coverage_test.go
|
||||||
|
- internal/tools/swagger2openapi/main.go
|
||||||
|
- internal/tools/swagger2openapi/main_test.go
|
||||||
|
- admin/openapi/admin.json
|
||||||
|
- admin/src/api/schema.d.ts
|
||||||
|
- modules/cabana/README.md
|
||||||
|
- docs/backend/forms.md
|
||||||
|
- docs/backend/lists-and-filters.md
|
||||||
|
- docs/database/attachments.md
|
||||||
|
|
||||||
|
key-decisions:
|
||||||
|
- "12.2-02: file routes on id 0 answer 404 when the controller does not declare create, as for a missing session key"
|
||||||
|
- "12.2-02: the save consumes (forgets) a file bind whose row is gone or already attached; unbinds of a file not attached to this owner are consumed the same way"
|
||||||
|
- "12.2-02: removing a file already pending removal is an idempotent 200; removing it through another record is 404"
|
||||||
|
- "12.2-02: preview thumbnails use imageWidth x imageHeight (one given means a square), else 240 x 240, in thumbOptions.mode or crop"
|
||||||
|
- "12.2-02: displayFormat is served only when format is set; the SPA otherwise formats by locale"
|
||||||
|
- "12.2-02: a fileupload field on a settings form fails boot (no controller owns file routes there); a settings datepicker gets the D-19 type check"
|
||||||
|
- "12.2-02: file limit and date bound messages use lagoon::validation lines with Laravel's English fallback and the field name (underscores as spaces) as :attribute"
|
||||||
|
|
||||||
|
patterns-established:
|
||||||
|
- "fileScope + findFile: (owner attached) OR (pending in the admin's own key), one query"
|
||||||
|
- "conformCase.raw: binary routes are checked for status, content type and headers, and admin.json must document the 200 as binary"
|
||||||
|
|
||||||
|
requirements-completed: [SC-1, SC-2, SC-4]
|
||||||
|
|
||||||
|
coverage:
|
||||||
|
- id: D1
|
||||||
|
description: "Upload to the create form (id 0), save with the same X-Session-Key: the file is attached and the binding gone; a foreign admin with the key sees and commits nothing"
|
||||||
|
requirement: SC-2
|
||||||
|
verification:
|
||||||
|
- kind: integration
|
||||||
|
ref: "modules/cabana/fileupload_smoke_test.go#TestFileuploadSmokeCreateCommit|TestFileuploadSmokeForeignAdmin"
|
||||||
|
status: pass
|
||||||
|
human_judgment: false
|
||||||
|
- id: D2
|
||||||
|
description: "Removing a pending upload deletes row and blob; attachOne replace and deferred removal delete the old file and blob on save"
|
||||||
|
requirement: SC-4
|
||||||
|
verification:
|
||||||
|
- kind: integration
|
||||||
|
ref: "modules/cabana/fileupload_smoke_test.go#TestFileuploadSmokeRemoveCancelsPending|TestFileuploadSmokeAttachOneReplace"
|
||||||
|
status: pass
|
||||||
|
human_judgment: false
|
||||||
|
- id: D3
|
||||||
|
description: "Protected download: another record's file and a public file are 404; an SVG is an octet-stream attachment with nosniff, private no-store and sandbox CSP"
|
||||||
|
requirement: SC-2
|
||||||
|
verification:
|
||||||
|
- kind: integration
|
||||||
|
ref: "modules/cabana/fileupload_smoke_test.go#TestProtectedFileSmoke"
|
||||||
|
status: pass
|
||||||
|
human_judgment: false
|
||||||
|
- id: D4
|
||||||
|
description: "Datepicker keys, format mapping and Go type check fail boot naming the file; date and datetime save (DATE kept, datetime in UTC); minDate enforced with 422"
|
||||||
|
requirement: SC-1
|
||||||
|
verification:
|
||||||
|
- kind: integration
|
||||||
|
ref: "modules/cabana/datepicker_smoke_test.go#TestDatepickerSmokeCompile|TestDatepickerSmokeTypeMismatch|TestDatepickerSmokeSave"
|
||||||
|
status: pass
|
||||||
|
human_judgment: false
|
||||||
|
- id: D5
|
||||||
|
description: "Every new route is inventoried, guarded, CSRF-walked, in admin.json and called once by the conformance suite"
|
||||||
|
verification:
|
||||||
|
- kind: integration
|
||||||
|
ref: "modules/cabana#TestPhase09PermissionMatrix|TestPhase09ContractInventory|TestPhase10CSRF|TestPhase10Coverage|TestPhase10OpenAPIConformance"
|
||||||
|
status: pass
|
||||||
|
- kind: other
|
||||||
|
ref: "scripts/check-admin-openapi.sh --check && scripts/check-admin-dist.sh"
|
||||||
|
status: pass
|
||||||
|
human_judgment: false
|
||||||
|
- id: D6
|
||||||
|
description: "Docs and READMEs name only existing identifiers; fonoteka.go unchanged and green"
|
||||||
|
verification:
|
||||||
|
- kind: other
|
||||||
|
ref: "go test ./cmd/summer -run TestDocsTree && go run ./cmd/summer docs:build --check && go -C ../fonoteka.go build ./... && go -C ../fonoteka.go vet ./... && go -C ../fonoteka.go test ./plugins/golem15/fonoteka -run Admin"
|
||||||
|
status: pass
|
||||||
|
human_judgment: false
|
||||||
|
|
||||||
|
duration: 28min
|
||||||
|
completed: 2026-10-02
|
||||||
|
status: complete
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 12.2 Plan 02: Admin API for fileupload and datepicker fields Summary
|
||||||
|
|
||||||
|
**WinterCMS-style `type: fileupload` with uploads and removals deferred against the SPA's X-Session-Key and applied inside the record's save transaction, seven parent-scoped file routes with hardened protected downloads, and `type: datepicker` with a boot-time Go type check and server-side date bounds**
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
|
||||||
|
- **Duration:** about 28 min
|
||||||
|
- **Started:** 2026-10-02T15:51:49Z
|
||||||
|
- **Completed:** 2026-10-02T16:20:00Z
|
||||||
|
- **Tasks:** 3
|
||||||
|
- **Files modified:** 29 (summercms.go only; fonoteka.go untouched)
|
||||||
|
|
||||||
|
## Accomplishments
|
||||||
|
|
||||||
|
- `type: fileupload` compiles exactly the D-08 keys. At boot the field is bound to the model's `attach.Relation` (the model must implement `attach.Owner` and `attach.HasRelations`). `multiple` and `protected` are derived from the relation, and `maxFiles` on attachOne or a `maxFilesize` above `http.body_limits.upload_bytes` stops boot.
|
||||||
|
- `X-Session-Key` is validated against `^[A-Za-z0-9_-]{32,128}$`. Every binding read and write is keyed by the key, the admin id from bouncer and the controller's morph type, so another admin's key matches nothing.
|
||||||
|
- The upload route caps the body at min(upload_bytes, maxFilesize + 64 KiB) and answers 413 `payload_too_large` past it. It accepts exactly one `file_data` part, stores it with `attach.Store` and binds it to the session. When the transaction rolls back it deletes the blob itself. Store refusals answer 422 on the field.
|
||||||
|
- The file list returns the attached files minus the session's pending removals plus its pending uploads. It sets `pending`, and gives URLs only for public relations. Caption (needs `useCaption`), remove (defers the removal, or cancels a pending upload at once) and reorder (attachMany only, the id set must match the visible files exactly) all resolve the file with one parent-scoped query.
|
||||||
|
- Protected download and thumb routes serve only `is_public=false` files, with nosniff, `private, no-store` and `default-src 'none'; sandbox`. They serve jpeg, png, gif and webp inline and send everything else as an octet-stream attachment.
|
||||||
|
- `commitDeferred` runs in `CRUDService.save` after `syncBelongsToMany` and before `FormAfterCreate`/`FormAfterUpdate`. It reads the bindings `FOR UPDATE`, attaches binds (on attachOne the replaced file is deleted first) and deletes unbound files, with blob deletes after commit. It then rechecks `maxFiles` and a required fileupload field. A 422 rolls back and leaves the bindings in place.
|
||||||
|
- `type: datepicker` compiles the D-20 keys and maps the PHP `format` to `displayFormat` with WinterCMS's momentFormat table; unmapped letters fail boot. Each mode is checked against its Go type (`time.Time`, `lagoon.Date` or `lagoon.TimeOfDay`, or a pointer to one). The field binds as a writable scalar, and the save rechecks `minDate`/`maxDate` with Laravel's `after_or_equal`/`before_or_equal` text.
|
||||||
|
- Lists accept `type: date` and `type: time`. `isListRelation` no longer classes Scanner/Valuer structs as relations.
|
||||||
|
|
||||||
|
## Task Commits
|
||||||
|
|
||||||
|
1. **Task 1: fileupload field, upload and list routes, commit on save** - `044e045` (feat)
|
||||||
|
2. **Task 2: remove, caption, reorder, protected download and thumb, commit rules** - `e54fd25` (feat)
|
||||||
|
3. **Task 3: datepicker field, bounds, date and time list columns** - `67d4c7f` (feat)
|
||||||
|
|
||||||
|
The measured count of 4 commits since `plan_head_before` includes `bbb8049 docs(13): capture phase context`. Another session committed it on master during this run, and it is not part of this plan.
|
||||||
|
|
||||||
|
## Files Created/Modified
|
||||||
|
|
||||||
|
- `modules/cabana/field_file.go`: fileupload compile and boot checks, `configBytes`, `FileItem`, `fileScope`/`parentFileScope`/`findFile`/`visibleFiles`, the seven handlers, the error mapping, `AdminFileCaptionRequest`
|
||||||
|
- `modules/cabana/deferred.go`: `SessionKeyHeader`, `sessionKeyFrom`, `commitDeferred` and the bind/unbind/delete helpers
|
||||||
|
- `modules/cabana/field_date.go`: datepicker compile, `momentFormat`, `checkDateType`, `compileDateFields`, `dateBoundDetails`
|
||||||
|
- `modules/cabana/crud.go`: `RecordInput.SessionKey`, commit and bounds in `save`, `datepicker` in `scalarFormField`, the bucket and translator on `CRUDService`
|
||||||
|
- `modules/cabana/http.go`: the routes, the `nestedGet` `files` case, the body limits read in `Activate`, the session key on create and update
|
||||||
|
- `modules/cabana/admin_openapi.go`: `FileMutationResult` and seven `AdminFile*` doc funcs; the `X-Session-Key` param on create and update
|
||||||
|
- `internal/tools/swagger2openapi`: formData to a multipart requestBody, `{file}` responses to binary (JSON errors kept JSON)
|
||||||
|
- Tests: the inventories, CSRF walk counts, the conformance fixture (`photos` public attachMany, `manual` protected attachOne, `released_on`/`starts_at` datepickers, a `released_on` date column, a memblob bucket) and the smoke tests
|
||||||
|
- Docs: cabana README (features, routes, API, configuration), forms (Date pickers, File uploads), lists-and-filters (date/time columns), attachments (Protected files in the admin)
|
||||||
|
|
||||||
|
## Decisions Made
|
||||||
|
|
||||||
|
See `key-decisions` in the frontmatter.
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
### Auto-fixed Issues
|
||||||
|
|
||||||
|
**1. [Rule 3 - Blocking] swagger2openapi could not express a multipart upload or a binary response**
|
||||||
|
- **Found during:** Tasks 1 and 2
|
||||||
|
- **Issue:** The converter left Swagger 2.0 `formData` parameters as invalid OpenAPI 3 parameters. It also put every response under every produced media type, so error envelopes would have been listed as octet-stream.
|
||||||
|
- **Fix:** Fold `formData` parameters into a `multipart/form-data` requestBody (a file becomes a binary string). Map a `{file}` response to `string/binary` under the operation's media types, and keep other schemas under JSON. Covered by `TestConvertFormData` and `TestConvertFileResponse`.
|
||||||
|
- **Files modified:** internal/tools/swagger2openapi/main.go, main_test.go
|
||||||
|
- **Commits:** 044e045, e54fd25
|
||||||
|
|
||||||
|
**2. [Rule 3 - Blocking] Phase 10 tests pin the set of state-changing routes**
|
||||||
|
- **Found during:** Task 1
|
||||||
|
- **Issue:** `TestPhase10CSRF` and `TestPhase10Coverage` expect exactly 11 unsafe routes.
|
||||||
|
- **Fix:** Added the upload, reorder, caption and remove routes; the count is now 15. All four routes go through `requireAjax`.
|
||||||
|
- **Commits:** 044e045, e54fd25
|
||||||
|
|
||||||
|
**3. [Rule 1 - Bug] Conformance fixture leaked files across tests**
|
||||||
|
- **Found during:** Task 1
|
||||||
|
- **Issue:** `newConformEnv` recreates the gadget table, so ids restart. Files attached to "gadget 1" by an earlier test then showed up on a new gadget 1.
|
||||||
|
- **Fix:** The fixture deletes the gadget morph type's `system_files` rows, unattached rows and `deferred_bindings` before each run.
|
||||||
|
- **Commit:** 044e045
|
||||||
|
|
||||||
|
**4. [Rule 2 - Missing critical] Settings forms**
|
||||||
|
- **Found during:** Task 1 and Task 3
|
||||||
|
- **Issue:** A settings screen has no controller that owns file routes, so a fileupload field there would render but could never upload. A settings datepicker skipped the D-19 type check.
|
||||||
|
- **Fix:** A fileupload field on a settings form fails boot, and a settings datepicker gets the same Go type check.
|
||||||
|
- **Files modified:** modules/cabana/settings.go
|
||||||
|
- **Commits:** 044e045, 67d4c7f
|
||||||
|
|
||||||
|
**5. [CLAUDE.md docs rule] Config keys cabana now reads**
|
||||||
|
- cabana now reads `http.body_limits.upload_bytes` and `default_bytes`, so both are listed in the cabana README Configuration table.
|
||||||
|
- **Commit:** 67d4c7f
|
||||||
|
|
||||||
|
### Not changed although listed
|
||||||
|
|
||||||
|
- `modules/cabana/partial_render.go` and `modules/cabana/query.go`: `isListRelation` is the one relation predicate that list_schema, filter_schema and query use, and it now excludes Scanner/Valuer structs. The struct-kind switches in partial_render.go walk view models for the security guard and never classify relations, so they needed no change.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Total deviations:** 5 auto-fixed (2 blocking, 1 bug, 1 missing critical, 1 docs rule)
|
||||||
|
**Impact on plan:** None add scope beyond the plan's truths; the converter change is needed for the documented upload and download routes.
|
||||||
|
|
||||||
|
## Issues Encountered
|
||||||
|
|
||||||
|
- GORM logs "record not found" for lagoon's binding lookups (`Take` in `findBinding`) during upload. This is noise from plan 01's code, not an error.
|
||||||
|
- `go test ./...` was run with FORCE_COLOR unset (known bonfire noise); the full suite passed at each task commit.
|
||||||
|
|
||||||
|
## Known Stubs
|
||||||
|
|
||||||
|
None.
|
||||||
|
|
||||||
|
## User Setup Required
|
||||||
|
|
||||||
|
None. To store uploads, the host application must publish a storage bucket (`attach.Publish`). Without one, the upload and protected-file routes answer 500 and log the cause.
|
||||||
|
|
||||||
|
## Next Phase Readiness
|
||||||
|
|
||||||
|
- Plan 03 can reuse `sessionKeyFrom`, `fileScope` (add a child scope constructor; the handler bodies take a resolved scope), and extend `commitDeferred` with relation bindings: it already reads the session's bindings in the save transaction, keyed by the controller's morph type.
|
||||||
|
- Plan 04 (SPA): the form schema carries `mode`, `displayFormat`, `minDate`, `maxDate`, `yearRange`, `firstDay`, `twelveHour`, `ignoreTimezone`, `fileTypes`, `mimeTypes`, `maxFilesize`, `maxFiles`, `imageWidth`, `imageHeight`, `thumbOptions`, `useCaption`, `prompt`, `multiple` and `protected`. A protected file has no `url` or `thumb_url`; the SPA builds the download and thumb route URLs.
|
||||||
|
- Plan 05: concurrency of two saves with one key (the `FOR UPDATE` read serializes them), maxFiles and required at save, and the 413 path deserve dedicated tests.
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
- Created files exist: field_file.go, field_date.go, deferred.go, fileupload_smoke_test.go, datepicker_smoke_test.go.
|
||||||
|
- Commits 044e045, e54fd25 and 67d4c7f are on master in summercms.go; fonoteka.go has no changes.
|
||||||
Reference in New Issue
Block a user