# lagoon

Postgres data layer: the shared GORM connection, per-plugin migrations, model helpers and file attachments.

`import "git.golem15.com/golem15/summercms/modules/lagoon"`

`import "git.golem15.com/golem15/summercms/modules/lagoon/attach"`

## Overview

`lagoon` is the WinterCMS models and migrations counterpart: it opens the one `*sql.DB` pool (pgx stdlib driver) that GORM and the rest of the application share, runs each plugin's gormigrate set in its own history table, and ports the Eloquent model conventions that WinterCMS plugins rely on, such as `$fillable`, `$hidden`, `$jsonable`, encrypted casts and Laravel-style validation rules. Its `attach` subpackage ports WinterCMS's `system_files` attachments, storing originals and lazily generated thumbnails in a gocloud.dev blob bucket. The application binary gets the migrate and key commands from `lagoon.RuntimeCommands`.

## Features

- One shared pool: `lagoon.Open`, `lagoon.Use` and `lagoon.OpenFromApp` return a `*sql.DB` and a `*gorm.DB` built on that same pool; `lagoon.Publish` makes both available on the `backpack.App`.
- Database-ready hooks: `lagoon.OnDatabase` runs a callback with the pool and GORM handle as soon as the database is published, immediately when it already is, otherwise when `lagoon.Publish` runs. Plugins register GORM callbacks through it from Boot, which runs before the `serve` command publishes the database.
- After-commit work: `lagoon.Transaction` runs a function in a transaction and then the callbacks registered with `lagoon.AfterCommit`, in order, only after the commit succeeds; a nested `lagoon.Transaction` is a savepoint whose callbacks are dropped with it when it fails. A nested `lagoon.Transaction` must be given the outer transaction's handle: given a root handle it returns an error without running its function, rather than open an independent transaction whose callbacks would wait on the outer one. A single-statement write for which GORM opens its own implicit transaction runs its callbacks from `lagoon:after_commit` once GORM commits, and never when the write fails. A callback registered inside a foreign plain GORM transaction is unsafe because Lagoon cannot observe its commit, so `lagoon.AfterCommit` warns and skips it. Outside a transaction, callbacks run immediately. The handle a supported callback receives always has an empty statement on the connection its work belongs to. A panicking callback is logged and never turns a committed write into an error.
- Per-plugin migrations: `lagoon.Migrate` runs the framework's `system_files` set (`attach.Migrations`), backend admin identity set (`lagoon.BackendAdminMigrations`) and job-queue set (`lagoon.QueueMigrations`: River's schema pinned at `lagoon.RiverSchemaVersion`, then the `lagoon.JobsTable` record table, under the `lagoon.QueueHistoryID` history), then every `pact.HasMigrations` set in plugin activation order, each in its own `summer_migrations_<plugin_id>` history table (`lagoon.HistoryTableName`). `lagoon.RollbackLast` and `lagoon.Status` cover rollback and history.
- Mass assignment: `lagoon.Fill` copies only allow-listed keys onto a model by GORM column name and silently drops the rest, logging each dropped key once outside production. A `json.Number` (from a decoder using `UseNumber`) fills integer, unsigned and float fields. A value that does not fit its column (a fraction, an exponent or an overflow for an integer field, or a value of the wrong type) is a `lagoon.FillTypeError` naming the key, so a caller can answer it as a validation failure on that field. `lagoon.HasFillable` and `lagoon.HasHidden` are the Go forms of `$fillable` and `$hidden`.
- Validation: `lagoon.Validate` accepts Laravel-style rule strings (`required`, `nullable`, `integer`, `numeric`, `between`, `min`, `max`, `in`, `unique`, `boolean`, `email`, `confirmed`, `different`, `mimes`) and returns a field-to-messages map, translated through phrasebook when a translator is given. Unknown rule tokens are an error. A failed numeric range reports the bound that failed: the `min` message below the lower bound, the `max` message above the upper one, and the numeric `between` message when the bound came from `between`.
- Request validation: `lagoon.ValidateRequest` reproduces Laravel 9 request validation for ported API endpoints, so a 422 body matches the PHP one message for message. It takes the decoded input and an ordered `lagoon.RequestRule` table (attribute names may hold `*` wildcards, expanded against the input to `posts.0.title`), runs the rules of each attribute in order and stops an attribute after a failed implicit rule (`required`, `present`, `filled`, `accepted`) or, under `bail`, after any failure. A non-implicit rule is skipped for an absent attribute, a blank string, a null value under `nullable` and an absent key under `sometimes`. Supported rules: `required`, `present`, `filled`, `accepted`, `nullable`, `sometimes`, `bail`, `array`, `string`, `integer`, `numeric`, `boolean`, `email` (PHP `FILTER_VALIDATE_EMAIL`, WinterCMS's default), `url`, `date`, `after`, `after_or_equal`, `before`, `before_or_equal` (a date, a relative word such as `tomorrow`, or another field), `exists:table,column`, `regex`, `not_regex`, `in`, `not_in`, `file`, `image`, `mimes`, `min`, `max`, `size` and `between`, plus closure rules built with `lagoon.CustomRule`. The size rules compare the number under `numeric` or `integer` (exactly, as decimals), the element count of an array, kilobytes of a `lagoon.UploadedFile`, and otherwise the length in characters, and pick the matching message. Messages come from the `lagoon::validation` catalog in the request locale; `lagoon.ErrorKeys` gives the attribute order of PHP's message bag.
- Safe ordering: `lagoon.OrderBy` appends an ORDER BY only for an allow-listed column and an `asc` or `desc` direction, and `lagoon.Collate` adds a validated `COLLATE` clause for language-specific text order (for example the ICU collation `pl-x-icu`); lagoon puts no requirement on the database's default locale.
- Pagination: `lagoon.Paginate` builds a `lagoon.Page` with `data` and `meta` (`current_page`, `last_page`, `per_page`, `total`).
- Column types: `lagoon.Encrypted` stores AES-256-GCM ciphertext under a key derived from `app.key`, decrypts with previous keys during rotation, and always redacts itself in JSON and string output; `lagoon.Jsonable` stores JSON as TEXT and keeps SQL NULL distinct from an empty value.
- Lifecycle and relations: hook interfaces matching GORM's native method names (`lagoon.HasBeforeCreate`, `lagoon.HasBeforeSave`, `lagoon.HasBeforeDelete`, `lagoon.HasAfterDelete`) plus `lagoon.HasBeforeValidate`; `lagoon.WithSoftDeleteCascade` runs a cascade inside the parent delete; `lagoon.RegisterJoinTable` wires pivot models with business columns.
- Imports from Laravel: `lagoon.DecryptLaravelPayload` decrypts Laravel `encrypted` payloads with the old application key, for one-off data imports.
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), public URLs (`attach.PublicURL` for any key, `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since WebP cannot be encoded), static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).

## Usage

Open the shared pool once at boot and publish it, so plugins and handlers reuse the same handles:

```go
sqlDB, gdb, err := lagoon.OpenFromApp(ctx, app)
if err != nil {
	return err
}
if err := lagoon.Publish(app, sqlDB, gdb); err != nil {
	return err
}
```

A model uses the column types and helpers directly:

```go
type Post struct {
	ID       uint                      `gorm:"column:id;primaryKey"`
	Title    string                    `gorm:"column:title"`
	Tags     lagoon.Jsonable[[]string] `gorm:"column:tags"`
	APIToken lagoon.Encrypted          `gorm:"column:api_token"`
}

func (Post) Fillable() []string { return []string{"title"} }

func createPost(ctx context.Context, gdb *gorm.DB, tr *phrasebook.Translator, input map[string]any) (map[string][]string, error) {
	var post Post
	rules := map[string]string{"title": "required|max:255|unique:posts"}
	if errs, err := lagoon.Validate(ctx, gdb, &post, rules, input, tr); err != nil || errs != nil {
		return errs, err
	}
	if err := lagoon.Fill(&post, post.Fillable(), input, false); err != nil {
		return nil, err
	}
	post.APIToken = lagoon.NewEncrypted("change-me")
	return nil, gdb.Create(&post).Error
}
```

A plugin registers its GORM callbacks from Boot through `lagoon.OnDatabase`, so they are installed whenever the database is published, and defers side effects until the write commits:

```go
func (p *Plugin) Boot(app *backpack.App) error {
	return lagoon.OnDatabase(app, func(_ *sql.DB, gdb *gorm.DB) error {
		return gdb.Callback().Create().After("gorm:after_create").Register("acme:post_created", func(db *gorm.DB) {
			if db.Error != nil {
				return
			}
			lagoon.AfterCommit(db.Statement.Context, db, func(ctx context.Context, db *gorm.DB) {
				// runs only once the insert is committed
			})
		})
	})
}

func publish(ctx context.Context, gdb *gorm.DB, post *Post) error {
	return lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
		return tx.Create(post).Error // acme:post_created work waits for this commit
	})
}
```

A plugin ships its schema as an ordered gormigrate set; `migrate` runs it after the framework sets:

```go
func (p *Plugin) Migrations() []*gormigrate.Migration {
	return []*gormigrate.Migration{{
		ID: "202601010001_create_posts",
		Migrate: func(tx *gorm.DB) error {
			return tx.Exec(`CREATE TABLE posts (id SERIAL PRIMARY KEY, title TEXT NOT NULL, tags TEXT, api_token TEXT)`).Error
		},
		Rollback: func(tx *gorm.DB) error {
			return tx.Exec(`DROP TABLE IF EXISTS posts`).Error
		},
	}}
}
```

## API reference

| Identifier | Description |
|------------|-------------|
| `lagoon.OpenFromApp` | Opens the shared pool from `database.dsn` and publishes the `app.key` encryption keys. |
| `lagoon.Open` | Opens and pings a DSN and returns the pool plus a GORM handle on it. |
| `lagoon.Use` | Returns a GORM handle on an existing pool after a ping. |
| `lagoon.Publish` | Stores the pool and GORM handle on the `backpack.App`, then runs the callbacks queued by `lagoon.OnDatabase`. |
| `lagoon.OnDatabase` | Runs a callback with the pool and GORM handle once the database is published. |
| `lagoon.Transaction` | Runs a function in a transaction (a savepoint when nested) and its `lagoon.AfterCommit` callbacks after the commit. |
| `lagoon.AfterCommit` | Registers work to run after the surrounding transaction commits. |
| `lagoon.AfterCommitCallback` | Name of the GORM callback, `lagoon:after_commit`, that runs single-statement after-commit work. |
| `lagoon.DSN` | Reads `database.dsn` from config. |
| `lagoon.Migrate` | Runs framework and plugin migrations in order. |
| `lagoon.RollbackLast` | Rolls back the last migration of one plugin. |
| `lagoon.Status` | Lists applied migration IDs per plugin as `lagoon.StatusRow` values. |
| `lagoon.HistoryTableName` | Returns the gormigrate history table for a plugin ID. |
| `lagoon.BackendAdminMigrations` | Creates the backend user, role and admin token blacklist tables, seeds the system roles and adds a case-insensitive unique index on `backend_users.email`. The index migration refuses to run while emails differ only in case and names the clashing logins. |
| `lagoon.QueueMigrations` | Migrates River's schema to `lagoon.RiverSchemaVersion` and creates the `summer_jobs` job record table. |
| `lagoon.QueueHistoryID` | History id of the job-queue set, `summercms.conga`. |
| `lagoon.JobsTable` | Name of the job record table, `summer_jobs`. |
| `lagoon.RiverSchemaVersion` | The pinned River schema version, 7. |
| `lagoon.RuntimeCommands` | Returns the migrate, migrate:rollback, migrate:status and key:generate commands. |
| `lagoon.KeyGenerateCommand` | Returns the key:generate command on its own. |
| `lagoon.LoadAppKey` | Decodes `app.key` and `app.previous_keys`. |
| `lagoon.PublishEncryptionKeys` | Installs the keys used by `lagoon.Encrypted` columns. |
| `lagoon.Encrypted` | Encrypted text column; `lagoon.Encrypted.Reveal` is the only plaintext accessor. |
| `lagoon.Jsonable` | Generic JSON-as-TEXT column with NULL tracking. |
| `lagoon.Fill` | Allow-listed mass assignment by column name. |
| `lagoon.FillTypeError` | Returned by `lagoon.Fill` when a requested value does not fit its column; `Key` names the column. |
| `lagoon.Validate` | Laravel-style rule validation with a `unique` database check. |
| `lagoon.ValidateRequest` | Laravel 9 request validation of decoded input against an ordered rule table; returns Laravel's errors object. |
| `lagoon.RequestRule` | One attribute of a request rule table: `Field` (wildcards allowed) and its ordered `Rules`. |
| `lagoon.Rule` | One parsed rule; `lagoon.Rule.Name` and `lagoon.Rule.Args` describe it. |
| `lagoon.ParseRules` | Parses a Laravel rule string into rules; keeps `regex:` patterns whole and panics on an unknown rule or an invalid pattern, so rule tables fail at boot. |
| `lagoon.In` | The `in` rule from a list of values, for values that contain commas or quotes (Laravel's `Rule::in`). |
| `lagoon.CustomRule` | A closure rule; its failure message is used as written. |
| `lagoon.UploadedFile` | An uploaded file for the file rules; `lagoon.UploadedFileFromHeader` adapts a `multipart.FileHeader`. |
| `lagoon.ErrorKeys` | The attributes of an errors object in Laravel's order for a rule table. |
| `lagoon.OrderBy` | Allow-listed ORDER BY, with an optional `lagoon.Collate`. |
| `lagoon.Collate` | Order option that sorts the column with a named PostgreSQL collation; the name is validated and quoted. |
| `lagoon.OrderOption` | Option type accepted by `lagoon.OrderBy`. |
| `lagoon.Paginate` | Builds a `lagoon.Page` with `lagoon.PageMeta`. |
| `lagoon.WithSoftDeleteCascade` | Runs a cascade inside the parent delete transaction. |
| `lagoon.RegisterJoinTable` | Registers a custom pivot model for a many-to-many field. |
| `lagoon.DecryptLaravelPayload` | Decrypts a Laravel AES-256-CBC payload for data imports. |
| `attach.File` | The `system_files` row model; `attach.File.URL` is the public URL of the original. |
| `attach.PublicURL` | Public URL of a blob key under `storage.uploads.public_path_prefix`. |
| `attach.Owner` | Implemented by models that own attachments; returns the stored morph type name. |
| `attach.OpenBucket` | Opens the uploads bucket from config. |
| `attach.Publish` | Stores the bucket on the `backpack.App`. |
| `attach.StaticHandler` | Serves stored files and thumbnails under a URL prefix. |
| `attach.StaticHandlerPublic` | `attach.StaticHandler` plus a 404 for rows that are not public. |
| `attach.DeleteForOwner` | Deletes an owner's attachment rows in a transaction and reports their blob keys. |
| `attach.DeleteKeys` | Deletes blobs, including thumbnails, after the transaction commits. |
| `attach.Migrations` | Creates the `system_files` table. |

## Configuration

Keys are read from the compass config; the `SUMMER_` environment overlay maps a double underscore to a dot, for example `SUMMER_DATABASE__DSN` to `database.dsn`.

| Key | Default | Controls |
|-----|---------|----------|
| `database.dsn` | none (required) | Postgres connection string used by `lagoon.OpenFromApp` and every database command. |
| `app.key` | none (required) | Base64 encoding of 32 random bytes; the source of the `lagoon.Encrypted` column key. Generate one with `key:generate`. |
| `app.previous_keys` | empty | List of earlier base64 keys still accepted when decrypting, for key rotation. |
| `storage.uploads.bucket_url` | none (required by `attach.OpenBucket`) | Uploads bucket URL, `file://` or `mem://`. |
| `storage.uploads.public_path_prefix` | `/storage/uploads` | URL prefix used when building public file and thumbnail URLs. |

```yaml
database:
  dsn: postgres://acme:<secret>@127.0.0.1:5432/acme?sslmode=disable
app:
  key: <base64 key from key:generate>
storage:
  uploads:
    bucket_url: file:///var/lib/acme/uploads
```

## CLI commands

`lagoon.RuntimeCommands` adds these commands to the application binary. All of them except key:generate open the database through `lagoon.OpenFromApp`, so they need `database.dsn` and `app.key`.

| Command | Flags | Description |
|---------|-------|-------------|
| `migrate` | none | Runs the framework migrations, then each plugin's migrations in dependency order. |
| `migrate:rollback` | `--plugin <id>` | Rolls back the last migration of the given plugin; without the flag, of the last activated plugin that has migrations. |
| `migrate:status` | none | Prints a table of plugin, history table and applied migration IDs. |
| `key:generate` | none | Prints a fresh base64 32-byte key for `app.key`; writes nothing. |

## Dependencies

- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [compass](../compass/README.md), [pact](../pact/README.md), [party](../party/README.md), [phrasebook](../phrasebook/README.md) (validation messages).
- Third-party: `gorm.io/gorm`, `gorm.io/driver/postgres`, `github.com/jackc/pgx/v5` (stdlib driver), `github.com/go-gormigrate/gormigrate/v2`, `github.com/go-playground/validator/v10`, `github.com/riverqueue/river` (its `rivermigrate` and `riverdriver/riverdatabasesql` packages, for the River schema in `lagoon.QueueMigrations`).
- Third-party, `attach` only: `gocloud.dev/blob` (file and memory drivers), `github.com/disintegration/imaging` (thumbnails), `golang.org/x/image/webp` (WebP decoding).
- Standard library: `database/sql`, `crypto/aes`, `crypto/cipher`, `crypto/hkdf`, `log/slog`, `image`, among others.

## Testing

```sh
go test ./modules/lagoon/...
```

The database tests in `lagoon` and `lagoon/attach` start a `postgres:16-alpine` container through testcontainers-go, so they need a running Docker daemon. They are skipped by `go test -short ./modules/lagoon/...`, which runs only the unit tests.