c3efbc3428
- lagoon.Fill converts a json.Number (from a UseNumber decoder, as cabana's save path uses) into integer, unsigned and float fields; a fraction or an overflow is an error - before this, saving a type: number field into an *int column was a 500 - README documents the conversion
168 lines
11 KiB
Markdown
168 lines
11 KiB
Markdown
# lagoon
|
|
|
|
Postgres data layer: the shared GORM connection, per-plugin migrations, model helpers and file attachments.
|
|
|
|
`import "git.golem15.com/golem15/summercms/modules/lagoon"`
|
|
|
|
`import "git.golem15.com/golem15/summercms/modules/lagoon/attach"`
|
|
|
|
## Overview
|
|
|
|
`lagoon` is the WinterCMS models and migrations counterpart: it opens the one `*sql.DB` pool (pgx stdlib driver) that GORM and the rest of the application share, runs each plugin's gormigrate set in its own history table, and ports the Eloquent model conventions that WinterCMS plugins rely on, such as `$fillable`, `$hidden`, `$jsonable`, encrypted casts and Laravel-style validation rules. Its `attach` subpackage ports WinterCMS's `system_files` attachments, storing originals and lazily generated thumbnails in a gocloud.dev blob bucket. The application binary gets the migrate and key commands from `lagoon.RuntimeCommands`.
|
|
|
|
## Features
|
|
|
|
- One shared pool: `lagoon.Open`, `lagoon.Use` and `lagoon.OpenFromApp` return a `*sql.DB` and a `*gorm.DB` built on that same pool; `lagoon.Publish` makes both available on the `backpack.App`.
|
|
- Database check at connect time: `lagoon.CheckLocale` refuses a database whose default collation is not the ICU `pl-PL` locale, so ordering matches the database default without per-query `COLLATE`.
|
|
- Per-plugin migrations: `lagoon.Migrate` runs the framework's `system_files` set (`attach.Migrations`) and backend admin identity set (`lagoon.BackendAdminMigrations`), then every `pact.HasMigrations` set in plugin activation order, each in its own `summer_migrations_<plugin_id>` history table (`lagoon.HistoryTableName`). `lagoon.RollbackLast` and `lagoon.Status` cover rollback and history.
|
|
- Mass assignment: `lagoon.Fill` copies only allow-listed keys onto a model by GORM column name and silently drops the rest, logging each dropped key once outside production. A `json.Number` (from a decoder using `UseNumber`) fills integer, unsigned and float fields, and a fraction or an overflow is an error. `lagoon.HasFillable` and `lagoon.HasHidden` are the Go forms of `$fillable` and `$hidden`.
|
|
- Validation: `lagoon.Validate` accepts Laravel-style rule strings (`required`, `nullable`, `integer`, `numeric`, `between`, `min`, `max`, `in`, `unique`, `boolean`, `email`, `confirmed`, `different`, `mimes`) and returns a field-to-messages map, translated through phrasebook when a translator is given. Unknown rule tokens are an error.
|
|
- Safe ordering: `lagoon.OrderBy` appends an ORDER BY only for an allow-listed column and an `asc` or `desc` direction.
|
|
- Pagination: `lagoon.Paginate` builds a `lagoon.Page` with `data` and `meta` (`current_page`, `last_page`, `per_page`, `total`).
|
|
- Column types: `lagoon.Encrypted` stores AES-256-GCM ciphertext under a key derived from `app.key`, decrypts with previous keys during rotation, and always redacts itself in JSON and string output; `lagoon.Jsonable` stores JSON as TEXT and keeps SQL NULL distinct from an empty value.
|
|
- Lifecycle and relations: hook interfaces matching GORM's native method names (`lagoon.HasBeforeCreate`, `lagoon.HasBeforeSave`, `lagoon.HasBeforeDelete`, `lagoon.HasAfterDelete`) plus `lagoon.HasBeforeValidate`; `lagoon.WithSoftDeleteCascade` runs a cascade inside the parent delete; `lagoon.RegisterJoinTable` wires pivot models with business columns.
|
|
- Imports from Laravel: `lagoon.DecryptLaravelPayload` decrypts Laravel `encrypted` payloads with the old application key, for one-off data imports.
|
|
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), on-demand thumbnails through `attach.File.Thumb`, static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
|
|
|
|
## Usage
|
|
|
|
Open the shared pool once at boot and publish it, so plugins and handlers reuse the same handles:
|
|
|
|
```go
|
|
sqlDB, gdb, err := lagoon.OpenFromApp(ctx, app)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if err := lagoon.Publish(app, sqlDB, gdb); err != nil {
|
|
return err
|
|
}
|
|
```
|
|
|
|
A model uses the column types and helpers directly:
|
|
|
|
```go
|
|
type Post struct {
|
|
ID uint `gorm:"column:id;primaryKey"`
|
|
Title string `gorm:"column:title"`
|
|
Tags lagoon.Jsonable[[]string] `gorm:"column:tags"`
|
|
APIToken lagoon.Encrypted `gorm:"column:api_token"`
|
|
}
|
|
|
|
func (Post) Fillable() []string { return []string{"title"} }
|
|
|
|
func createPost(ctx context.Context, gdb *gorm.DB, tr *phrasebook.Translator, input map[string]any) (map[string][]string, error) {
|
|
var post Post
|
|
rules := map[string]string{"title": "required|max:255|unique:posts"}
|
|
if errs, err := lagoon.Validate(ctx, gdb, &post, rules, input, tr); err != nil || errs != nil {
|
|
return errs, err
|
|
}
|
|
if err := lagoon.Fill(&post, post.Fillable(), input, false); err != nil {
|
|
return nil, err
|
|
}
|
|
post.APIToken = lagoon.NewEncrypted("change-me")
|
|
return nil, gdb.Create(&post).Error
|
|
}
|
|
```
|
|
|
|
A plugin ships its schema as an ordered gormigrate set; `migrate` runs it after the framework sets:
|
|
|
|
```go
|
|
func (p *Plugin) Migrations() []*gormigrate.Migration {
|
|
return []*gormigrate.Migration{{
|
|
ID: "202601010001_create_posts",
|
|
Migrate: func(tx *gorm.DB) error {
|
|
return tx.Exec(`CREATE TABLE posts (id SERIAL PRIMARY KEY, title TEXT NOT NULL, tags TEXT, api_token TEXT)`).Error
|
|
},
|
|
Rollback: func(tx *gorm.DB) error {
|
|
return tx.Exec(`DROP TABLE IF EXISTS posts`).Error
|
|
},
|
|
}}
|
|
}
|
|
```
|
|
|
|
## API reference
|
|
|
|
| Identifier | Description |
|
|
|------------|-------------|
|
|
| `lagoon.OpenFromApp` | Opens the shared pool from `database.dsn` and publishes the `app.key` encryption keys. |
|
|
| `lagoon.Open` | Opens and pings a DSN, checks the locale and returns the pool plus a GORM handle on it. |
|
|
| `lagoon.Use` | Returns a GORM handle on an existing pool after the same checks. |
|
|
| `lagoon.Publish` | Stores the pool and GORM handle on the `backpack.App`. |
|
|
| `lagoon.DSN` | Reads `database.dsn` from config. |
|
|
| `lagoon.CheckLocale` | Fails unless the database default locale is ICU `pl-PL`. |
|
|
| `lagoon.Migrate` | Runs framework and plugin migrations in order. |
|
|
| `lagoon.RollbackLast` | Rolls back the last migration of one plugin. |
|
|
| `lagoon.Status` | Lists applied migration IDs per plugin as `lagoon.StatusRow` values. |
|
|
| `lagoon.HistoryTableName` | Returns the gormigrate history table for a plugin ID. |
|
|
| `lagoon.BackendAdminMigrations` | Creates the backend user, role and admin token blacklist tables and seeds the system roles. |
|
|
| `lagoon.RuntimeCommands` | Returns the migrate, migrate:rollback, migrate:status and key:generate commands. |
|
|
| `lagoon.KeyGenerateCommand` | Returns the key:generate command on its own. |
|
|
| `lagoon.LoadAppKey` | Decodes `app.key` and `app.previous_keys`. |
|
|
| `lagoon.PublishEncryptionKeys` | Installs the keys used by `lagoon.Encrypted` columns. |
|
|
| `lagoon.Encrypted` | Encrypted text column; `lagoon.Encrypted.Reveal` is the only plaintext accessor. |
|
|
| `lagoon.Jsonable` | Generic JSON-as-TEXT column with NULL tracking. |
|
|
| `lagoon.Fill` | Allow-listed mass assignment by column name. |
|
|
| `lagoon.Validate` | Laravel-style rule validation with a `unique` database check. |
|
|
| `lagoon.OrderBy` | Allow-listed ORDER BY. |
|
|
| `lagoon.Paginate` | Builds a `lagoon.Page` with `lagoon.PageMeta`. |
|
|
| `lagoon.WithSoftDeleteCascade` | Runs a cascade inside the parent delete transaction. |
|
|
| `lagoon.RegisterJoinTable` | Registers a custom pivot model for a many-to-many field. |
|
|
| `lagoon.DecryptLaravelPayload` | Decrypts a Laravel AES-256-CBC payload for data imports. |
|
|
| `attach.File` | The `system_files` row model. |
|
|
| `attach.Owner` | Implemented by models that own attachments; returns the stored morph type name. |
|
|
| `attach.OpenBucket` | Opens the uploads bucket from config. |
|
|
| `attach.Publish` | Stores the bucket on the `backpack.App`. |
|
|
| `attach.StaticHandler` | Serves stored files and thumbnails under a URL prefix. |
|
|
| `attach.StaticHandlerPublic` | `attach.StaticHandler` plus a 404 for rows that are not public. |
|
|
| `attach.DeleteForOwner` | Deletes an owner's attachment rows in a transaction and reports their blob keys. |
|
|
| `attach.DeleteKeys` | Deletes blobs, including thumbnails, after the transaction commits. |
|
|
| `attach.Migrations` | Creates the `system_files` table. |
|
|
|
|
## Configuration
|
|
|
|
Keys are read from the compass config; the `SUMMER_` environment overlay maps a double underscore to a dot, for example `SUMMER_DATABASE__DSN` to `database.dsn`.
|
|
|
|
| Key | Default | Controls |
|
|
|-----|---------|----------|
|
|
| `database.dsn` | none (required) | Postgres connection string used by `lagoon.OpenFromApp` and every database command. |
|
|
| `app.key` | none (required) | Base64 encoding of 32 random bytes; the source of the `lagoon.Encrypted` column key. Generate one with `key:generate`. |
|
|
| `app.previous_keys` | empty | List of earlier base64 keys still accepted when decrypting, for key rotation. |
|
|
| `storage.uploads.bucket_url` | none (required by `attach.OpenBucket`) | Uploads bucket URL, `file://` or `mem://`. |
|
|
| `storage.uploads.public_path_prefix` | `/storage/uploads` | URL prefix used when building public file and thumbnail URLs. |
|
|
|
|
```yaml
|
|
database:
|
|
dsn: postgres://acme:<secret>@127.0.0.1:5432/acme?sslmode=disable
|
|
app:
|
|
key: <base64 key from key:generate>
|
|
storage:
|
|
uploads:
|
|
bucket_url: file:///var/lib/acme/uploads
|
|
```
|
|
|
|
## CLI commands
|
|
|
|
`lagoon.RuntimeCommands` adds these commands to the application binary. All of them except key:generate open the database through `lagoon.OpenFromApp`, so they need `database.dsn` and `app.key`.
|
|
|
|
| Command | Flags | Description |
|
|
|---------|-------|-------------|
|
|
| `migrate` | none | Runs the framework migrations, then each plugin's migrations in dependency order. |
|
|
| `migrate:rollback` | `--plugin <id>` | Rolls back the last migration of the given plugin; without the flag, of the last activated plugin that has migrations. |
|
|
| `migrate:status` | none | Prints a table of plugin, history table and applied migration IDs. |
|
|
| `key:generate` | none | Prints a fresh base64 32-byte key for `app.key`; writes nothing. |
|
|
|
|
## Dependencies
|
|
|
|
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [compass](../compass/README.md), [pact](../pact/README.md), [party](../party/README.md), [phrasebook](../phrasebook/README.md) (validation messages).
|
|
- Third-party: `gorm.io/gorm`, `gorm.io/driver/postgres`, `github.com/jackc/pgx/v5` (stdlib driver), `github.com/go-gormigrate/gormigrate/v2`, `github.com/go-playground/validator/v10`.
|
|
- Third-party, `attach` only: `gocloud.dev/blob` (file and memory drivers), `github.com/disintegration/imaging` (thumbnails).
|
|
- Standard library: `database/sql`, `crypto/aes`, `crypto/cipher`, `crypto/hkdf`, `log/slog`, `image`, among others.
|
|
|
|
## Testing
|
|
|
|
```sh
|
|
go test ./modules/lagoon/...
|
|
```
|
|
|
|
The database tests in `lagoon` and `lagoon/attach` start a `postgres:16-alpine` container, initialised with the ICU `pl-PL` locale, through testcontainers-go, so they need a running Docker daemon. They are skipped by `go test -short ./modules/lagoon/...`, which runs only the unit tests.
|