golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(12.2-01): add deferred bindings, guarded upload store and purge

Browse Source 
- deferred_bindings migration set under summercms.deferred with backend_user_id
- lagoon.DeferredBind/Unbind/Bindings/Forget/Slaves scoped by DeferredKey
- lagoon.PurgeDeferred with SKIP LOCKED batches and after-commit blob deletes
- attach.Store with the ported image guard, extension and MIME limits
- attach.Relation, attach.HasRelations, attach.BlobKeys, File.ThumbKey
- lagoon README and attachments docs
This commit is contained in:
Jakub Zych
2026-10-02 17:36:43 +02:00
parent 79e2a43095
commit 19f4cf8232
14 changed files with 1280 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
modules/lagoon/README.md
View File

@@ -15,7 +15,7 @@ Postgres data layer: the shared GORM connection, per-plugin migrations, model he
- One shared pool: `lagoon.Open`, `lagoon.Use` and `lagoon.OpenFromApp` return a `*sql.DB` and a `*gorm.DB` built on that same pool; `lagoon.Publish` makes both available on the `backpack.App`.
- Database-ready hooks: `lagoon.OnDatabase` runs a callback with the pool and GORM handle as soon as the database is published, immediately when it already is, otherwise when `lagoon.Publish` runs. Plugins register GORM callbacks through it from Boot, which runs before the `serve` command publishes the database.
- After-commit work: `lagoon.Transaction` runs a function in a transaction and then the callbacks registered with `lagoon.AfterCommit`, in order, only after the commit succeeds; a nested `lagoon.Transaction` is a savepoint whose callbacks are dropped with it when it fails. A nested `lagoon.Transaction` must be given the outer transaction's handle: given a root handle it returns an error without running its function, rather than open an independent transaction whose callbacks would wait on the outer one. A single-statement write for which GORM opens its own implicit transaction runs its callbacks from `lagoon:after_commit` once GORM commits, and never when the write fails. A callback registered inside a foreign plain GORM transaction is unsafe because Lagoon cannot observe its commit, so `lagoon.AfterCommit` warns and skips it. Outside a transaction, callbacks run immediately. The handle a supported callback receives always has an empty statement on the connection its work belongs to. A panicking callback is logged and never turns a committed write into an error.
- Per-plugin migrations: `lagoon.Migrate` runs the framework's `system_files` set (`attach.Migrations`), backend admin identity set (`lagoon.BackendAdminMigrations`) and job-queue set (`lagoon.QueueMigrations`: River's schema pinned at `lagoon.RiverSchemaVersion`, then the `lagoon.JobsTable` record table, under the `lagoon.QueueHistoryID` history), then every `pact.HasMigrations` set in plugin activation order, each in its own `summer_migrations_<plugin_id>` history table (`lagoon.HistoryTableName`). `lagoon.RollbackLast` and `lagoon.Status` cover rollback and history.
- Per-plugin migrations: `lagoon.Migrate` runs the framework's `system_files` set (`attach.Migrations`), backend admin identity set (`lagoon.BackendAdminMigrations`), `deferred_bindings` set (`lagoon.DeferredBindingMigrations`, under the `lagoon.DeferredHistoryID` history) and job-queue set (`lagoon.QueueMigrations`: River's schema pinned at `lagoon.RiverSchemaVersion`, then the `lagoon.JobsTable` record table, under the `lagoon.QueueHistoryID` history), then every `pact.HasMigrations` set in plugin activation order, each in its own `summer_migrations_<plugin_id>` history table (`lagoon.HistoryTableName`). `lagoon.RollbackLast` and `lagoon.Status` cover rollback and history.
- Mass assignment: `lagoon.Fill` copies only allow-listed keys onto a model by GORM column name and silently drops the rest, logging each dropped key once outside production. A `json.Number` (from a decoder using `UseNumber`) fills integer, unsigned and float fields. A value that does not fit its column (a fraction, an exponent or an overflow for an integer field, or a value of the wrong type) is a `lagoon.FillTypeError` naming the key, so a caller can answer it as a validation failure on that field. `lagoon.HasFillable` and `lagoon.HasHidden` are the Go forms of `$fillable` and `$hidden`.
- Validation: `lagoon.Validate` accepts Laravel-style rule strings (`required`, `nullable`, `integer`, `numeric`, `between`, `min`, `max`, `in`, `unique`, `boolean`, `email`, `confirmed`, `different`, `mimes`) and returns a field-to-messages map, translated through phrasebook when a translator is given. Unknown rule tokens are an error. A failed numeric range reports the bound that failed: the `min` message below the lower bound, the `max` message above the upper one, and the numeric `between` message when the bound came from `between`.
- Request validation: `lagoon.ValidateRequest` reproduces Laravel 9 request validation for ported API endpoints, so a 422 body matches the PHP one message for message. It takes the decoded input and an ordered `lagoon.RequestRule` table (attribute names may hold `*` wildcards, expanded against the input to `posts.0.title`), runs the rules of each attribute in order and stops an attribute after a failed implicit rule (`required`, `present`, `filled`, `accepted`) or, under `bail`, after any failure. A non-implicit rule is skipped for an absent attribute, a blank string, a null value under `nullable` and an absent key under `sometimes`. Supported rules: `required`, `present`, `filled`, `accepted`, `nullable`, `sometimes`, `bail`, `array`, `string`, `integer`, `numeric`, `boolean`, `email` (PHP `FILTER_VALIDATE_EMAIL`, WinterCMS's default), `url`, `date`, `after`, `after_or_equal`, `before`, `before_or_equal` (a date, a relative word such as `tomorrow`, or another field), `exists:table,column`, `regex`, `not_regex`, `in`, `not_in`, `file`, `image`, `mimes`, `min`, `max`, `size` and `between`, plus closure rules built with `lagoon.CustomRule`. The size rules compare the number under `numeric` or `integer` (exactly, as decimals), the element count of an array, kilobytes of a `lagoon.UploadedFile`, and otherwise the length in characters, and pick the matching message. Messages come from the `lagoon::validation` catalog in the request locale; `lagoon.ErrorKeys` gives the attribute order of PHP's message bag.
@@ -23,8 +23,9 @@ Postgres data layer: the shared GORM connection, per-plugin migrations, model he
- Pagination: `lagoon.Paginate` builds a `lagoon.Page` with `data` and `meta` (`current_page`, `last_page`, `per_page`, `total`).
- Column types: `lagoon.Encrypted` stores AES-256-GCM ciphertext under a key derived from `app.key`, decrypts with previous keys during rotation, and always redacts itself in JSON and string output; `lagoon.Jsonable` stores JSON as TEXT and keeps SQL NULL distinct from an empty value.
- Lifecycle and relations: hook interfaces matching GORM's native method names (`lagoon.HasBeforeCreate`, `lagoon.HasBeforeSave`, `lagoon.HasBeforeDelete`, `lagoon.HasAfterDelete`) plus `lagoon.HasBeforeValidate`; `lagoon.WithSoftDeleteCascade` runs a cascade inside the parent delete; `lagoon.RegisterJoinTable` wires pivot models with business columns.
- Deferred binding: WinterCMS's `deferred_bindings` table holds the uploads and related-record changes of a form whose record is not saved yet. Every operation takes a `lagoon.DeferredKey` (the form's session key, the owning backend admin's id and the master record's morph type from `lagoon.MorphType`) and never reads or changes another admin's rows, since each row stores `backend_user_id`. `lagoon.DeferredBind` and `lagoon.DeferredUnbind` port WinterCMS's duplicate and cancel rules: a repeated bind writes nothing, and an unbind of a slave with a pending bind deletes that bind and returns it so the caller can remove what it created. `lagoon.DeferredBindings` reads and locks a session's bindings for the save that commits them, `lagoon.DeferredForget` deletes them once applied, and `lagoon.DeferredSlaves` is the subquery a list uses to include pending rows. A child created under deferral carries the `lagoon.DeferredEnvelope` (`{"created":true,"pivot":{...}}`) in `pivot_data`. `lagoon.PurgeDeferred` removes expired bindings: it deletes an unattached `system_files` row a bind points at, and its blobs only after the commit, deletes a child only when its binding carries the created envelope, keeps records that were only linked, and locks each batch with `FOR UPDATE SKIP LOCKED`.
- Imports from Laravel: `lagoon.DecryptLaravelPayload` decrypts Laravel `encrypted` payloads with the old application key, for one-off data imports.
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), public URLs (`attach.PublicURL` for any key, `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since WebP cannot be encoded; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), public URLs (`attach.PublicURL` for any key, `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since WebP cannot be encoded; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming and, in image mode, the `attach.IsAllowedImage` content guard), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
## Usage
@@ -127,6 +128,21 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
| `lagoon.QueueHistoryID` | History id of the job-queue set, `summercms.conga`. |
| `lagoon.JobsTable` | Name of the job record table, `summer_jobs`. |
| `lagoon.RiverSchemaVersion` | The pinned River schema version, 7. |
| `lagoon.DeferredBindingMigrations` | Creates WinterCMS's `deferred_bindings` table plus the `backend_user_id` owner column. |
| `lagoon.DeferredHistoryID` | History id of the deferred-binding set, `summercms.deferred`. |
| `lagoon.DeferredBinding` | The `deferred_bindings` row model; `lagoon.DeferredBinding.Envelope` decodes its `pivot_data`. |
| `lagoon.DeferredKey` | Session key, admin id and master type that scope every deferred-binding operation. |
| `lagoon.DeferredEnvelope` | The framework's `pivot_data` shape: `Created` marks a child created under deferral, `Pivot` holds pivot values. |
| `lagoon.DeferredFileType` | The `slave_type` of a binding that points at a `system_files` row. |
| `lagoon.MorphType` | The `master_type` or `slave_type` string of a model: its `attach.Owner` morph name, else its table name. |
| `lagoon.DeferredBind` | Records a pending bind; a repeat writes nothing and a pending unbind of the same slave is cancelled. |
| `lagoon.DeferredUnbind` | Records a pending unbind, or cancels a pending bind of the same slave and returns it. |
| `lagoon.DeferredBindings` | Reads and locks a session's bindings for the given relation fields, in id order. |
| `lagoon.DeferredForget` | Deletes applied bindings by id. |
| `lagoon.DeferredSlaves` | Subquery of a session's bound or unbound slave ids, for list queries. |
| `lagoon.PurgeDeferred` | Removes expired bindings, their unattached files (blobs after commit) and the children created under deferral. |
| `lagoon.PurgeOptions` | Cut-off time and created-child model resolver for `lagoon.PurgeDeferred`. |
| `lagoon.PurgeResult` | Counts of deleted bindings, files and children and of skipped bindings. |
| `lagoon.RuntimeCommands` | Returns the migrate, migrate:rollback, migrate:status and key:generate commands. |
| `lagoon.KeyGenerateCommand` | Returns the key:generate command on its own. |
| `lagoon.LoadAppKey` | Decodes `app.key` and `app.previous_keys`. |
@@ -162,6 +178,22 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
| `attach.DeleteForOwner` | Deletes an owner's attachment rows in a transaction and reports their blob keys. |
| `attach.DeleteKeys` | Deletes blobs, including thumbnails, after the transaction commits. |
| `attach.Migrations` | Creates the `system_files` table. |
| `attach.Store` | Stores an upload as an unattached `system_files` row with `sort_order` equal to its id, after the type, size and image checks. |
| `attach.Upload` | The client file name, body and public flag of one upload. |
| `attach.Limits` | Size limit, allowed extensions, allowed MIME types and image mode for `attach.Store`. |
| `attach.ErrTooLarge` | Returned by `attach.Store` for a body over `MaxBytes`. |
| `attach.ErrFileType` | Returned by `attach.Store` for a missing, malformed or disallowed extension. |
| `attach.ErrMIMEType` | Returned by `attach.Store` when the content type matches no `MIMETypes` entry. |
| `attach.ErrNotImage` | Returned by `attach.Store` in image mode for content that is not an allowed image. |
| `attach.DefaultImageExtensions` | Image-mode extensions when none are given: jpg, jpeg, png, gif, webp. |
| `attach.DefaultFileExtensions` | File-mode extensions when none are given: WinterCMS's default list without the script-capable types. |
| `attach.AllowedImageMIMEs` | The sniffed content types the image guard accepts. |
| `attach.IsAllowedImage` | The image guard: sniffed type, decoded header and a pixel ceiling, failing closed. |
| `attach.MaxImagePixels` | The image guard's pixel ceiling, 4096 by 4096. |
| `attach.Relation` | One attachOne or attachMany relation: `Name`, `Many` and `Public`. |
| `attach.HasRelations` | Implemented by an owner model that declares its attachment relations. |
| `attach.BlobKeys` | The original's key and the thumbnail prefix of a file, for `attach.DeleteKeys`. |
| `attach.File.ThumbKey` | Blob key of a lazily generated thumbnail, for serving a protected file's thumbnail without a public URL. |
## Configuration
@@ -191,7 +223,7 @@ storage:
| Command | Flags | Description |
|---------|-------|-------------|
| `migrate` | none | Runs the framework migrations, then each plugin's migrations in dependency order. |
| `migrate` | none | Runs the framework migrations (`system_files`, the backend admin tables, `deferred_bindings` and the job queue), then each plugin's migrations in dependency order. |
| `migrate:rollback` | `--plugin <id>` | Rolls back the last migration of the given plugin; without the flag, of the last activated plugin that has migrations. |
| `migrate:status` | none | Prints a table of plugin, history table and applied migration IDs. |
| `key:generate` | none | Prints a fresh base64 32-byte key for `app.key`; writes nothing. |