feat(lagoon): per-query collation for OrderBy, drop the database locale check

- lagoon.OrderBy takes variadic lagoon.OrderOption values; lagoon.Collate(name)
  emits a validated, double-quoted COLLATE clause (e.g. "pl-x-icu")
- remove the exported CheckLocale and the ICU pl-PL check from Open and Use
- framework test containers and per-test databases are plain PostgreSQL
- lagoon README, root README and docs pages drop the locale requirement;
  queries-and-pagination gains a "Sorting with a collation" section
  backed by ExampleCollate
This commit is contained in:
Jakub Zych
2026-10-01 09:51:03 +02:00
parent 565ce982d9
commit 037dc53030
22 changed files with 344 additions and 168 deletions

View File

@@ -8,7 +8,7 @@ order: 10
A WinterCMS model extends Eloquent and describes its behaviour with properties such as `$fillable`, `$hidden` and `$jsonable`. A SummerCMS model is a plain GORM struct in the plugin's `models` package, and [lagoon](../../modules/lagoon/README.md) supplies the Eloquent conventions that GORM does not have: allow-listed mass assignment, JSON and encrypted columns, Laravel-style validation and pagination.
The data layer supports PostgreSQL only. `lagoon.Open` refuses a database whose default collation is not the ICU `pl-PL` locale (`lagoon.CheckLocale`), so text ordering is the same in every query without a `COLLATE` clause. Create the database with that locale, as shown in [Installation](../setup/installation.md).
The data layer supports PostgreSQL only and puts no requirement on the database's default locale. A list that must sort text in one language's order passes a collation to `lagoon.OrderBy`, as described in [Queries and pagination](queries-and-pagination.md#sorting-with-a-collation).
## Defining a model

View File

@@ -39,7 +39,32 @@ fmt.Println(err)
Answer the error as a validation failure (422). The column must match an allow-list entry exactly, so list the qualified name (`acme_blog_posts.title`) when the query joins another table.
`lagoon.OrderBy` never adds a `COLLATE` clause. Text sorts by the database's ICU `pl-PL` default collation, which lagoon checks when it connects.
## Sorting with a collation
Text sorts by the database's default collation unless you pass `lagoon.Collate`. A list that must follow one language's alphabet (Polish puts Ł between L and M, for example) passes `lagoon.Collate("pl-x-icu")`, and `lagoon.OrderBy` adds a `COLLATE` clause for that column only. ICU collations named `<language>-x-icu` exist in any PostgreSQL built with ICU support, which includes the official Docker images, so the database needs no special locale:
```go src=modules/lagoon/example_test.go#ExampleCollate
// A dry-run handle shows the SQL without a database.
db, _ := gorm.Open(postgres.New(postgres.Config{DSN: "host=127.0.0.1"}), &gorm.Config{DryRun: true, DisableAutomaticPing: true})
allowed := []string{"title", "views"}
// Polish alphabetical order, whatever the database's default locale.
q, err := lagoon.OrderBy(db.Model(&Post{}), "title", "asc", allowed, lagoon.Collate("pl-x-icu"))
if err != nil {
fmt.Println(err)
return
}
var posts []Post
fmt.Println(q.Find(&posts).Statement.SQL.String())
_, err = lagoon.OrderBy(db, "title", "asc", allowed, lagoon.Collate(`pl-x-icu" ASC, (SELECT 1) --`))
fmt.Println(err)
// Output:
// SELECT * FROM "acme_blog_posts" WHERE "acme_blog_posts"."deleted_at" IS NULL ORDER BY title COLLATE "pl-x-icu" ASC
// lagoon: order collation "pl-x-icu\" ASC, (SELECT 1) --" is not a valid collation name
```
The collation name is validated (ASCII letters, digits, `_`, `-`, `.` and `@`, at most 63 bytes) and quoted as an identifier; anything else is an error, and no SQL is built. An index only helps that ORDER BY when it is built with the same collation.
## Pagination

View File

@@ -100,10 +100,9 @@ func stopPostgres() {
}
}
// icuDatabase creates a database for one test with the ICU pl-PL locale
// lagoon requires, and drops it when the test ends. It returns the pool
// opened on it and its DSN.
func icuDatabase(t *testing.T) (*sql.DB, string) {
// testDatabase creates a database for one test and drops it when the test
// ends. It returns the pool opened on it and its DSN.
func testDatabase(t *testing.T) (*sql.DB, string) {
t.Helper()
if testing.Short() {
t.Skip("requires testcontainers postgres")
@@ -113,7 +112,7 @@ func icuDatabase(t *testing.T) (*sql.DB, string) {
}
name := "blog_" + strings.ToLower(strings.NewReplacer("/", "_", "-", "_").Replace(t.Name()))
quoted := `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
if _, err := pgAdmin.ExecContext(t.Context(), `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`); err != nil {
if _, err := pgAdmin.ExecContext(t.Context(), `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8'`); err != nil {
t.Fatalf("create %s: %v", name, err)
}
u, err := url.Parse(pgDSN)
@@ -133,11 +132,11 @@ func icuDatabase(t *testing.T) (*sql.DB, string) {
return db, dsn
}
// migrated activates acme.blog, migrates a fresh ICU database and publishes
// migrated activates acme.blog, migrates a fresh database and publishes
// it on the application, as the serve command does at start-up.
func migrated(t *testing.T) (*backpack.App, party.Plugin, *gorm.DB) {
t.Helper()
sqlDB, _ := icuDatabase(t)
sqlDB, _ := testDatabase(t)
gdb, err := lagoon.Use(t.Context(), sqlDB)
if err != nil {
t.Fatalf("lagoon.Use: %v", err)

View File

@@ -31,10 +31,9 @@ Most plugin code runs without a database. Build a container with `backpack.New`,
## Database tests
SummerCMS supports PostgreSQL only, so database tests run against real PostgreSQL rather than an SQLite stand-in. The framework's own tests start a `postgres:16-alpine` container through testcontainers-go and create the database with the ICU `pl-PL` locale that lagoon checks for when it connects. Follow the same pattern in plugin tests:
SummerCMS supports PostgreSQL only, so database tests run against real PostgreSQL rather than an SQLite stand-in. The framework's own tests start a `postgres:16-alpine` container through testcontainers-go and create a fresh database per test. Follow the same pattern in plugin tests:
- skip the test when `testing.Short` reports true;
- create the database with `LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`, or lagoon refuses the connection;
- migrate a fresh database per test with `lagoon.Migrate`, so tests do not depend on each other.
Docker must be running for these tests.

View File

@@ -11,7 +11,7 @@ SummerCMS is a Go module. An application requires it, lists its plugins in a `su
## Requirements
- Go 1.27.
- PostgreSQL 16 for any application that uses the data layer. The database's default locale must be the ICU `pl-PL` locale, which lagoon checks when it connects.
- PostgreSQL 16 for any application that uses the data layer.
- Docker, only for the integration tests that start PostgreSQL or Mailpit containers.
You do not need Node.js to build an application or these docs. It is needed only when you work on the admin SPA itself.
@@ -74,10 +74,10 @@ summer build
## Create the database
The commands that touch data open PostgreSQL. Create a database with the locale lagoon checks for:
The commands that touch data open PostgreSQL. Create a database for the example:
```sql
CREATE DATABASE hello TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL';
CREATE DATABASE hello;
```
## Configure the application
@@ -117,7 +117,6 @@ From the application directory, `summer migrate`, `summer migrate:status` and `s
> Known issues in the current framework:
>
> - `examples/hello` ships no `http.body_limits` configuration, so `serve` and `route:list` fail with `surf: config http.body_limits.default_bytes is required` until you add `config/http.yaml` as shown above. The same gap makes the example's `TestTypedItemRoute` fail.
> - lagoon refuses any database whose default locale is not ICU `pl-PL`.
> - The committed `examples/hello/main.go` is older than what `summer build` generates now, so building the example leaves that file modified. Restore it with `git checkout -- examples/hello/main.go` if you do not intend to commit it.
## Next steps