feat(lagoon): per-query collation for OrderBy, drop the database locale check
- lagoon.OrderBy takes variadic lagoon.OrderOption values; lagoon.Collate(name) emits a validated, double-quoted COLLATE clause (e.g. "pl-x-icu") - remove the exported CheckLocale and the ICU pl-PL check from Open and Use - framework test containers and per-test databases are plain PostgreSQL - lagoon README, root README and docs pages drop the locale requirement; queries-and-pagination gains a "Sorting with a collation" section backed by ExampleCollate
This commit is contained in:
@@ -8,7 +8,7 @@ order: 10
|
||||
|
||||
A WinterCMS model extends Eloquent and describes its behaviour with properties such as `$fillable`, `$hidden` and `$jsonable`. A SummerCMS model is a plain GORM struct in the plugin's `models` package, and [lagoon](../../modules/lagoon/README.md) supplies the Eloquent conventions that GORM does not have: allow-listed mass assignment, JSON and encrypted columns, Laravel-style validation and pagination.
|
||||
|
||||
The data layer supports PostgreSQL only. `lagoon.Open` refuses a database whose default collation is not the ICU `pl-PL` locale (`lagoon.CheckLocale`), so text ordering is the same in every query without a `COLLATE` clause. Create the database with that locale, as shown in [Installation](../setup/installation.md).
|
||||
The data layer supports PostgreSQL only and puts no requirement on the database's default locale. A list that must sort text in one language's order passes a collation to `lagoon.OrderBy`, as described in [Queries and pagination](queries-and-pagination.md#sorting-with-a-collation).
|
||||
|
||||
## Defining a model
|
||||
|
||||
|
||||
@@ -39,7 +39,32 @@ fmt.Println(err)
|
||||
|
||||
Answer the error as a validation failure (422). The column must match an allow-list entry exactly, so list the qualified name (`acme_blog_posts.title`) when the query joins another table.
|
||||
|
||||
`lagoon.OrderBy` never adds a `COLLATE` clause. Text sorts by the database's ICU `pl-PL` default collation, which lagoon checks when it connects.
|
||||
## Sorting with a collation
|
||||
|
||||
Text sorts by the database's default collation unless you pass `lagoon.Collate`. A list that must follow one language's alphabet (Polish puts Ł between L and M, for example) passes `lagoon.Collate("pl-x-icu")`, and `lagoon.OrderBy` adds a `COLLATE` clause for that column only. ICU collations named `<language>-x-icu` exist in any PostgreSQL built with ICU support, which includes the official Docker images, so the database needs no special locale:
|
||||
|
||||
```go src=modules/lagoon/example_test.go#ExampleCollate
|
||||
// A dry-run handle shows the SQL without a database.
|
||||
db, _ := gorm.Open(postgres.New(postgres.Config{DSN: "host=127.0.0.1"}), &gorm.Config{DryRun: true, DisableAutomaticPing: true})
|
||||
allowed := []string{"title", "views"}
|
||||
|
||||
// Polish alphabetical order, whatever the database's default locale.
|
||||
q, err := lagoon.OrderBy(db.Model(&Post{}), "title", "asc", allowed, lagoon.Collate("pl-x-icu"))
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
var posts []Post
|
||||
fmt.Println(q.Find(&posts).Statement.SQL.String())
|
||||
|
||||
_, err = lagoon.OrderBy(db, "title", "asc", allowed, lagoon.Collate(`pl-x-icu" ASC, (SELECT 1) --`))
|
||||
fmt.Println(err)
|
||||
// Output:
|
||||
// SELECT * FROM "acme_blog_posts" WHERE "acme_blog_posts"."deleted_at" IS NULL ORDER BY title COLLATE "pl-x-icu" ASC
|
||||
// lagoon: order collation "pl-x-icu\" ASC, (SELECT 1) --" is not a valid collation name
|
||||
```
|
||||
|
||||
The collation name is validated (ASCII letters, digits, `_`, `-`, `.` and `@`, at most 63 bytes) and quoted as an identifier; anything else is an error, and no SQL is built. An index only helps that ORDER BY when it is built with the same collation.
|
||||
|
||||
## Pagination
|
||||
|
||||
|
||||
@@ -100,10 +100,9 @@ func stopPostgres() {
|
||||
}
|
||||
}
|
||||
|
||||
// icuDatabase creates a database for one test with the ICU pl-PL locale
|
||||
// lagoon requires, and drops it when the test ends. It returns the pool
|
||||
// opened on it and its DSN.
|
||||
func icuDatabase(t *testing.T) (*sql.DB, string) {
|
||||
// testDatabase creates a database for one test and drops it when the test
|
||||
// ends. It returns the pool opened on it and its DSN.
|
||||
func testDatabase(t *testing.T) (*sql.DB, string) {
|
||||
t.Helper()
|
||||
if testing.Short() {
|
||||
t.Skip("requires testcontainers postgres")
|
||||
@@ -113,7 +112,7 @@ func icuDatabase(t *testing.T) (*sql.DB, string) {
|
||||
}
|
||||
name := "blog_" + strings.ToLower(strings.NewReplacer("/", "_", "-", "_").Replace(t.Name()))
|
||||
quoted := `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
|
||||
if _, err := pgAdmin.ExecContext(t.Context(), `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`); err != nil {
|
||||
if _, err := pgAdmin.ExecContext(t.Context(), `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8'`); err != nil {
|
||||
t.Fatalf("create %s: %v", name, err)
|
||||
}
|
||||
u, err := url.Parse(pgDSN)
|
||||
@@ -133,11 +132,11 @@ func icuDatabase(t *testing.T) (*sql.DB, string) {
|
||||
return db, dsn
|
||||
}
|
||||
|
||||
// migrated activates acme.blog, migrates a fresh ICU database and publishes
|
||||
// migrated activates acme.blog, migrates a fresh database and publishes
|
||||
// it on the application, as the serve command does at start-up.
|
||||
func migrated(t *testing.T) (*backpack.App, party.Plugin, *gorm.DB) {
|
||||
t.Helper()
|
||||
sqlDB, _ := icuDatabase(t)
|
||||
sqlDB, _ := testDatabase(t)
|
||||
gdb, err := lagoon.Use(t.Context(), sqlDB)
|
||||
if err != nil {
|
||||
t.Fatalf("lagoon.Use: %v", err)
|
||||
|
||||
@@ -31,10 +31,9 @@ Most plugin code runs without a database. Build a container with `backpack.New`,
|
||||
|
||||
## Database tests
|
||||
|
||||
SummerCMS supports PostgreSQL only, so database tests run against real PostgreSQL rather than an SQLite stand-in. The framework's own tests start a `postgres:16-alpine` container through testcontainers-go and create the database with the ICU `pl-PL` locale that lagoon checks for when it connects. Follow the same pattern in plugin tests:
|
||||
SummerCMS supports PostgreSQL only, so database tests run against real PostgreSQL rather than an SQLite stand-in. The framework's own tests start a `postgres:16-alpine` container through testcontainers-go and create a fresh database per test. Follow the same pattern in plugin tests:
|
||||
|
||||
- skip the test when `testing.Short` reports true;
|
||||
- create the database with `LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`, or lagoon refuses the connection;
|
||||
- migrate a fresh database per test with `lagoon.Migrate`, so tests do not depend on each other.
|
||||
|
||||
Docker must be running for these tests.
|
||||
|
||||
@@ -11,7 +11,7 @@ SummerCMS is a Go module. An application requires it, lists its plugins in a `su
|
||||
## Requirements
|
||||
|
||||
- Go 1.27.
|
||||
- PostgreSQL 16 for any application that uses the data layer. The database's default locale must be the ICU `pl-PL` locale, which lagoon checks when it connects.
|
||||
- PostgreSQL 16 for any application that uses the data layer.
|
||||
- Docker, only for the integration tests that start PostgreSQL or Mailpit containers.
|
||||
|
||||
You do not need Node.js to build an application or these docs. It is needed only when you work on the admin SPA itself.
|
||||
@@ -74,10 +74,10 @@ summer build
|
||||
|
||||
## Create the database
|
||||
|
||||
The commands that touch data open PostgreSQL. Create a database with the locale lagoon checks for:
|
||||
The commands that touch data open PostgreSQL. Create a database for the example:
|
||||
|
||||
```sql
|
||||
CREATE DATABASE hello TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL';
|
||||
CREATE DATABASE hello;
|
||||
```
|
||||
|
||||
## Configure the application
|
||||
@@ -117,7 +117,6 @@ From the application directory, `summer migrate`, `summer migrate:status` and `s
|
||||
> Known issues in the current framework:
|
||||
>
|
||||
> - `examples/hello` ships no `http.body_limits` configuration, so `serve` and `route:list` fail with `surf: config http.body_limits.default_bytes is required` until you add `config/http.yaml` as shown above. The same gap makes the example's `TestTypedItemRoute` fail.
|
||||
> - lagoon refuses any database whose default locale is not ICU `pl-PL`.
|
||||
> - The committed `examples/hello/main.go` is older than what `summer build` generates now, so building the example leaves that file modified. Restore it with `git checkout -- examples/hello/main.go` if you do not intend to commit it.
|
||||
|
||||
## Next steps
|
||||
|
||||
Reference in New Issue
Block a user