summercms/modules/lagoon

lagoon

Postgres data layer: the shared GORM connection, per-plugin migrations, model helpers and file attachments.

import "git.golem15.com/golem15/summercms/modules/lagoon"

import "git.golem15.com/golem15/summercms/modules/lagoon/attach"

Overview

lagoon is the WinterCMS models and migrations counterpart: it opens the one *sql.DB pool (pgx stdlib driver) that GORM and the rest of the application share, runs each plugin's gormigrate set in its own history table, and ports the Eloquent model conventions that WinterCMS plugins rely on, such as $fillable, $hidden, $jsonable, encrypted casts and Laravel-style validation rules. Its attach subpackage ports WinterCMS's system_files attachments, storing originals and lazily generated thumbnails in a gocloud.dev blob bucket. The application binary gets the migrate and key commands from lagoon.RuntimeCommands.

Features

• One shared pool: lagoon.Open, lagoon.Use and lagoon.OpenFromApp return a *sql.DB and a *gorm.DB built on that same pool; lagoon.Publish makes both available on the backpack.App.
• Database-ready hooks: lagoon.OnDatabase runs a callback with the pool and GORM handle as soon as the database is published, immediately when it already is, otherwise when lagoon.Publish runs. Plugins register GORM callbacks through it from Boot, which runs before the serve command publishes the database.
• After-commit work: lagoon.Transaction runs a function in a transaction and then the callbacks registered with lagoon.AfterCommit, in order, only after the commit succeeds; a nested lagoon.Transaction is a savepoint whose callbacks are dropped with it when it fails. A nested lagoon.Transaction must be given the outer transaction's handle: given a root handle it returns an error without running its function, rather than open an independent transaction whose callbacks would wait on the outer one. A single-statement write for which GORM opens its own implicit transaction runs its callbacks from lagoon:after_commit once GORM commits, and never when the write fails. A callback registered inside a foreign plain GORM transaction is unsafe because Lagoon cannot observe its commit, so lagoon.AfterCommit warns and skips it. Outside a transaction, callbacks run immediately. The handle a supported callback receives always has an empty statement on the connection its work belongs to. A panicking callback is logged and never turns a committed write into an error.
• Per-plugin migrations: lagoon.Migrate runs the framework's system_files set (attach.Migrations), backend admin identity set (lagoon.BackendAdminMigrations), deferred_bindings set (lagoon.DeferredBindingMigrations, under the lagoon.DeferredHistoryID history) and job-queue set (lagoon.QueueMigrations: River's schema pinned at lagoon.RiverSchemaVersion, then the lagoon.JobsTable record table, under the lagoon.QueueHistoryID history), then every pact.HasMigrations set in plugin activation order, each in its own summer_migrations_<plugin_id> history table (lagoon.HistoryTableName). lagoon.RollbackLast and lagoon.Status cover rollback and history.
• Mass assignment: lagoon.Fill copies only allow-listed keys onto a model by GORM column name and silently drops the rest, logging each dropped key once outside production. A json.Number (from a decoder using UseNumber) fills integer, unsigned and float fields. A value that does not fit its column (a fraction, an exponent or an overflow for an integer field, or a value of the wrong type) is a lagoon.FillTypeError naming the key, so a caller can answer it as a validation failure on that field. lagoon.HasFillable and lagoon.HasHidden are the Go forms of $fillable and $hidden.
• Validation: lagoon.Validate accepts Laravel-style rule strings (required, nullable, integer, numeric, between, min, max, in, unique, boolean, email, confirmed, different, mimes) and returns a field-to-messages map, translated through phrasebook when a translator is given. Unknown rule tokens are an error. A failed numeric range reports the bound that failed: the min message below the lower bound, the max message above the upper one, and the numeric between message when the bound came from between.
• Request validation: lagoon.ValidateRequest reproduces Laravel 9 request validation for ported API endpoints, so a 422 body matches the PHP one message for message. It takes the decoded input and an ordered lagoon.RequestRule table (attribute names may hold * wildcards, expanded against the input to posts.0.title), runs the rules of each attribute in order and stops an attribute after a failed implicit rule (required, present, filled, accepted) or, under bail, after any failure. A non-implicit rule is skipped for an absent attribute, a blank string, a null value under nullable and an absent key under sometimes. Supported rules: required, present, filled, accepted, nullable, sometimes, bail, array, string, integer, numeric, boolean, email (PHP FILTER_VALIDATE_EMAIL, WinterCMS's default), url, date, after, after_or_equal, before, before_or_equal (a date, a relative word such as tomorrow, or another field), exists:table,column, regex, not_regex, in, not_in, file, image, mimes, min, max, size and between, plus closure rules built with lagoon.CustomRule. The size rules compare the number under numeric or integer (exactly, as decimals), the element count of an array, kilobytes of a lagoon.UploadedFile, and otherwise the length in characters, and pick the matching message. Messages come from the lagoon::validation catalog in the request locale; lagoon.ErrorKeys gives the attribute order of PHP's message bag.
• Safe ordering: lagoon.OrderBy appends an ORDER BY only for an allow-listed column and an asc or desc direction, and lagoon.Collate adds a validated COLLATE clause for language-specific text order (for example the ICU collation pl-x-icu); lagoon puts no requirement on the database's default locale.
• Pagination: lagoon.Paginate builds a lagoon.Page with data and meta (current_page, last_page, per_page, total).
• Column types: lagoon.Encrypted stores AES-256-GCM ciphertext under a key derived from app.key, decrypts with previous keys during rotation, and always redacts itself in JSON and string output; lagoon.Jsonable stores JSON as TEXT and keeps SQL NULL distinct from an empty value.
• Lifecycle and relations: hook interfaces matching GORM's native method names (lagoon.HasBeforeCreate, lagoon.HasBeforeSave, lagoon.HasBeforeDelete, lagoon.HasAfterDelete) plus lagoon.HasBeforeValidate; lagoon.WithSoftDeleteCascade runs a cascade inside the parent delete; lagoon.RegisterJoinTable wires pivot models with business columns.
• Deferred binding: WinterCMS's deferred_bindings table holds the uploads and related-record changes of a form whose record is not saved yet. Every operation takes a lagoon.DeferredKey (the form's session key, the owning backend admin's id and the master record's morph type from lagoon.MorphType) and never reads or changes another admin's rows, since each row stores backend_user_id. lagoon.DeferredBind and lagoon.DeferredUnbind port WinterCMS's duplicate and cancel rules: a repeated bind writes nothing, and an unbind of a slave with a pending bind deletes that bind and returns it so the caller can remove what it created. lagoon.DeferredBindings reads and locks a session's bindings for the save that commits them, lagoon.DeferredForget deletes them once applied, and lagoon.DeferredSlaves is the subquery a list uses to include pending rows. A child created under deferral carries the lagoon.DeferredEnvelope ({"created":true,"pivot":{...}}) in pivot_data. lagoon.PurgeDeferred removes expired bindings: it deletes an unattached system_files row a bind points at, and its blobs only after the commit, deletes a child only when its binding carries the created envelope, keeps records that were only linked, and locks each batch with FOR UPDATE SKIP LOCKED.
• Imports from Laravel: lagoon.DecryptLaravelPayload decrypts Laravel encrypted payloads with the old application key, for one-off data imports.
• Attachments (attach): the attach.File model for system_files rows, WinterCMS-compatible partitioned storage keys (attach.BlobKey, attach.PartitionDirectory), public URLs (attach.PublicURL for any key, attach.File.URL for an original, matching WinterCMS's File::getPath() under the WinterCMS layout), on-demand thumbnails through attach.File.Thumb for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its .webp name, since WebP cannot be encoded; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, attach.BrokenImagePNG, as its thumbnail, as File::makeThumb does), storing uploads through attach.Store (a server-generated disk name, an extension allow-list with attach.DefaultImageExtensions and attach.DefaultFileExtensions as defaults, a MIME filter, a size limit enforced while streaming and, in image mode, the attach.IsAllowedImage content guard), attachment relation declarations (attach.Relation, attach.HasRelations), static serving with an optional is_public gate (attach.StaticHandlerPublic), and a two-phase delete that removes blobs only after the database transaction commits (attach.DeleteForOwner, attach.DeleteKeys).

Usage

Open the shared pool once at boot and publish it, so plugins and handlers reuse the same handles:

sqlDB, gdb, err := lagoon.OpenFromApp(ctx, app)
if err != nil {
	return err
}
if err := lagoon.Publish(app, sqlDB, gdb); err != nil {
	return err
}

A model uses the column types and helpers directly:

type Post struct {
	ID       uint                      `gorm:"column:id;primaryKey"`
	Title    string                    `gorm:"column:title"`
	Tags     lagoon.Jsonable[[]string] `gorm:"column:tags"`
	APIToken lagoon.Encrypted          `gorm:"column:api_token"`
}

func (Post) Fillable() []string { return []string{"title"} }

func createPost(ctx context.Context, gdb *gorm.DB, tr *phrasebook.Translator, input map[string]any) (map[string][]string, error) {
	var post Post
	rules := map[string]string{"title": "required|max:255|unique:posts"}
	if errs, err := lagoon.Validate(ctx, gdb, &post, rules, input, tr); err != nil || errs != nil {
		return errs, err
	}
	if err := lagoon.Fill(&post, post.Fillable(), input, false); err != nil {
		return nil, err
	}
	post.APIToken = lagoon.NewEncrypted("change-me")
	return nil, gdb.Create(&post).Error
}

A plugin registers its GORM callbacks from Boot through lagoon.OnDatabase, so they are installed whenever the database is published, and defers side effects until the write commits:

func (p *Plugin) Boot(app *backpack.App) error {
	return lagoon.OnDatabase(app, func(_ *sql.DB, gdb *gorm.DB) error {
		return gdb.Callback().Create().After("gorm:after_create").Register("acme:post_created", func(db *gorm.DB) {
			if db.Error != nil {
				return
			}
			lagoon.AfterCommit(db.Statement.Context, db, func(ctx context.Context, db *gorm.DB) {
				// runs only once the insert is committed
			})
		})
	})
}

func publish(ctx context.Context, gdb *gorm.DB, post *Post) error {
	return lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
		return tx.Create(post).Error // acme:post_created work waits for this commit
	})
}

A plugin ships its schema as an ordered gormigrate set; migrate runs it after the framework sets:

func (p *Plugin) Migrations() []*gormigrate.Migration {
	return []*gormigrate.Migration{{
		ID: "202601010001_create_posts",
		Migrate: func(tx *gorm.DB) error {
			return tx.Exec(`CREATE TABLE posts (id SERIAL PRIMARY KEY, title TEXT NOT NULL, tags TEXT, api_token TEXT)`).Error
		},
		Rollback: func(tx *gorm.DB) error {
			return tx.Exec(`DROP TABLE IF EXISTS posts`).Error
		},
	}}
}

API reference

Identifier	Description
lagoon.OpenFromApp	Opens the shared pool from database.dsn and publishes the app.key encryption keys.
lagoon.Open	Opens and pings a DSN and returns the pool plus a GORM handle on it.
lagoon.Use	Returns a GORM handle on an existing pool after a ping.
lagoon.Publish	Stores the pool and GORM handle on the backpack.App, then runs the callbacks queued by lagoon.OnDatabase.
lagoon.OnDatabase	Runs a callback with the pool and GORM handle once the database is published.
lagoon.Transaction	Runs a function in a transaction (a savepoint when nested) and its lagoon.AfterCommit callbacks after the commit.
lagoon.AfterCommit	Registers work to run after the surrounding transaction commits.
lagoon.AfterCommitCallback	Name of the GORM callback, lagoon:after_commit, that runs single-statement after-commit work.
lagoon.DSN	Reads database.dsn from config.
lagoon.Migrate	Runs framework and plugin migrations in order.
lagoon.RollbackLast	Rolls back the last migration of one plugin.
lagoon.Status	Lists applied migration IDs per plugin as lagoon.StatusRow values.
lagoon.HistoryTableName	Returns the gormigrate history table for a plugin ID.
lagoon.BackendAdminMigrations	Creates the backend user, role and admin token blacklist tables, seeds the system roles and adds a case-insensitive unique index on backend_users.email. The index migration refuses to run while emails differ only in case and names the clashing logins.
lagoon.QueueMigrations	Migrates River's schema to lagoon.RiverSchemaVersion and creates the summer_jobs job record table.
lagoon.QueueHistoryID	History id of the job-queue set, summercms.conga.
lagoon.JobsTable	Name of the job record table, summer_jobs.
lagoon.RiverSchemaVersion	The pinned River schema version, 7.
lagoon.DeferredBindingMigrations	Creates WinterCMS's deferred_bindings table plus the backend_user_id owner column.
lagoon.DeferredHistoryID	History id of the deferred-binding set, summercms.deferred.
lagoon.DeferredBinding	The deferred_bindings row model; lagoon.DeferredBinding.Envelope decodes its pivot_data.
lagoon.DeferredKey	Session key, admin id and master type that scope every deferred-binding operation.
lagoon.DeferredEnvelope	The framework's pivot_data shape: Created marks a child created under deferral, Pivot holds pivot values.
lagoon.DeferredFileType	The slave_type of a binding that points at a system_files row.
lagoon.MorphType	The master_type or slave_type string of a model: its attach.Owner morph name, else its table name.
lagoon.DeferredBind	Records a pending bind; a repeat writes nothing and a pending unbind of the same slave is cancelled.
lagoon.DeferredUnbind	Records a pending unbind, or cancels a pending bind of the same slave and returns it.
lagoon.DeferredBindings	Reads and locks a session's bindings for the given relation fields, in id order.
lagoon.DeferredForget	Deletes applied bindings by id.
lagoon.DeferredSlaves	Subquery of a session's bound or unbound slave ids, for list queries.
lagoon.PurgeDeferred	Removes expired bindings, their unattached files (blobs after commit) and the children created under deferral.
lagoon.PurgeOptions	Cut-off time and created-child model resolver for lagoon.PurgeDeferred.
lagoon.PurgeResult	Counts of deleted bindings, files and children and of skipped bindings.
lagoon.RuntimeCommands	Returns the migrate, migrate:rollback, migrate:status and key:generate commands.
lagoon.KeyGenerateCommand	Returns the key:generate command on its own.
lagoon.LoadAppKey	Decodes app.key and app.previous_keys.
lagoon.PublishEncryptionKeys	Installs the keys used by lagoon.Encrypted columns.
lagoon.Encrypted	Encrypted text column; lagoon.Encrypted.Reveal is the only plaintext accessor.
lagoon.Jsonable	Generic JSON-as-TEXT column with NULL tracking.
lagoon.Fill	Allow-listed mass assignment by column name.
lagoon.FillTypeError	Returned by lagoon.Fill when a requested value does not fit its column; Key names the column.
lagoon.Validate	Laravel-style rule validation with a unique database check.
lagoon.ValidateRequest	Laravel 9 request validation of decoded input against an ordered rule table; returns Laravel's errors object.
lagoon.RequestRule	One attribute of a request rule table: Field (wildcards allowed) and its ordered Rules.
lagoon.Rule	One parsed rule; lagoon.Rule.Name and lagoon.Rule.Args describe it.
lagoon.ParseRules	Parses a Laravel rule string into rules; keeps regex: patterns whole and panics on an unknown rule or an invalid pattern, so rule tables fail at boot.
lagoon.In	The in rule from a list of values, for values that contain commas or quotes (Laravel's Rule::in).
lagoon.CustomRule	A closure rule; its failure message is used as written.
lagoon.UploadedFile	An uploaded file for the file rules; lagoon.UploadedFileFromHeader adapts a multipart.FileHeader.
lagoon.ErrorKeys	The attributes of an errors object in Laravel's order for a rule table.
lagoon.OrderBy	Allow-listed ORDER BY, with an optional lagoon.Collate.
lagoon.Collate	Order option that sorts the column with a named PostgreSQL collation; the name is validated and quoted.
lagoon.OrderOption	Option type accepted by lagoon.OrderBy.
lagoon.Paginate	Builds a lagoon.Page with lagoon.PageMeta.
lagoon.WithSoftDeleteCascade	Runs a cascade inside the parent delete transaction.
lagoon.RegisterJoinTable	Registers a custom pivot model for a many-to-many field.
lagoon.DecryptLaravelPayload	Decrypts a Laravel AES-256-CBC payload for data imports.
attach.File	The system_files row model; attach.File.URL is the public URL of the original.
attach.PublicURL	Public URL of a blob key under storage.uploads.public_path_prefix.
attach.BrokenImagePNG	WinterCMS's broken-image picture, stored as the thumbnail of an unusable original.
attach.Owner	Implemented by models that own attachments; returns the stored morph type name.
attach.OpenBucket	Opens the uploads bucket from config.
attach.Publish	Stores the bucket on the backpack.App.
attach.StaticHandler	Serves stored files and thumbnails under a URL prefix.
attach.StaticHandlerPublic	attach.StaticHandler plus a 404 for rows that are not public.
attach.DeleteForOwner	Deletes an owner's attachment rows in a transaction and reports their blob keys.
attach.DeleteKeys	Deletes blobs, including thumbnails, after the transaction commits.
attach.Migrations	Creates the system_files table.
attach.Store	Stores an upload as an unattached system_files row with sort_order equal to its id, after the type, size and image checks.
attach.Upload	The client file name, body and public flag of one upload.
attach.Limits	Size limit, allowed extensions, allowed MIME types and image mode for attach.Store.
attach.ErrTooLarge	Returned by attach.Store for a body over MaxBytes.
attach.ErrFileType	Returned by attach.Store for a missing, malformed or disallowed extension.
attach.ErrMIMEType	Returned by attach.Store when the content type matches no MIMETypes entry.
attach.ErrNotImage	Returned by attach.Store in image mode for content that is not an allowed image.
attach.DefaultImageExtensions	Image-mode extensions when none are given: jpg, jpeg, png, gif, webp.
attach.DefaultFileExtensions	File-mode extensions when none are given: WinterCMS's default list without the script-capable types.
attach.AllowedImageMIMEs	The sniffed content types the image guard accepts.
attach.IsAllowedImage	The image guard: sniffed type, decoded header and a pixel ceiling, failing closed.
attach.MaxImagePixels	The image guard's pixel ceiling, 4096 by 4096.
attach.Relation	One attachOne or attachMany relation: Name, Many and Public.
attach.HasRelations	Implemented by an owner model that declares its attachment relations.
attach.BlobKeys	The original's key and the thumbnail prefix of a file, for attach.DeleteKeys.
attach.File.ThumbKey	Blob key of a lazily generated thumbnail, for serving a protected file's thumbnail without a public URL.

Configuration

Keys are read from the compass config; the SUMMER_ environment overlay maps a double underscore to a dot, for example SUMMER_DATABASE__DSN to database.dsn.

Key	Default	Controls
database.dsn	none (required)	Postgres connection string used by lagoon.OpenFromApp and every database command.
app.key	none (required)	Base64 encoding of 32 random bytes; the source of the lagoon.Encrypted column key. Generate one with key:generate.
app.previous_keys	empty	List of earlier base64 keys still accepted when decrypting, for key rotation.
storage.uploads.bucket_url	none (required by attach.OpenBucket)	Uploads bucket URL, file:// or mem://.
storage.uploads.public_path_prefix	/storage/uploads	URL prefix used when building public file and thumbnail URLs.

database:
  dsn: postgres://acme:<secret>@127.0.0.1:5432/acme?sslmode=disable
app:
  key: <base64 key from key:generate>
storage:
  uploads:
    bucket_url: file:///var/lib/acme/uploads

CLI commands

lagoon.RuntimeCommands adds these commands to the application binary. All of them except key:generate open the database through lagoon.OpenFromApp, so they need database.dsn and app.key.

Command	Flags	Description
migrate	none	Runs the framework migrations (system_files, the backend admin tables, deferred_bindings and the job queue), then each plugin's migrations in dependency order.
migrate:rollback	--plugin <id>	Rolls back the last migration of the given plugin; without the flag, of the last activated plugin that has migrations.
migrate:status	none	Prints a table of plugin, history table and applied migration IDs.
key:generate	none	Prints a fresh base64 32-byte key for app.key; writes nothing.

Dependencies

• SummerCMS modules: backpack, bonfire, compass, pact, party, phrasebook (validation messages).
• Third-party: gorm.io/gorm, gorm.io/driver/postgres, github.com/jackc/pgx/v5 (stdlib driver), github.com/go-gormigrate/gormigrate/v2, github.com/go-playground/validator/v10, github.com/riverqueue/river (its rivermigrate and riverdriver/riverdatabasesql packages, for the River schema in lagoon.QueueMigrations).
• Third-party, attach only: gocloud.dev/blob (file and memory drivers), github.com/disintegration/imaging (thumbnails), golang.org/x/image/webp (WebP decoding).
• Standard library: database/sql, crypto/aes, crypto/cipher, crypto/hkdf, log/slog, image, among others.

Testing

go test ./modules/lagoon/...

The database tests in lagoon and lagoon/attach start a postgres:16-alpine container through testcontainers-go, so they need a running Docker daemon. They are skipped by go test -short ./modules/lagoon/..., which runs only the unit tests.