feat(lagoon): per-query collation for OrderBy, drop the database locale check
- lagoon.OrderBy takes variadic lagoon.OrderOption values; lagoon.Collate(name) emits a validated, double-quoted COLLATE clause (e.g. "pl-x-icu") - remove the exported CheckLocale and the ICU pl-PL check from Open and Use - framework test containers and per-test databases are plain PostgreSQL - lagoon README, root README and docs pages drop the locale requirement; queries-and-pagination gains a "Sorting with a collation" section backed by ExampleCollate
This commit is contained in:
@@ -12,7 +12,7 @@ SummerCMS is a content management framework for Go, inspired by WinterCMS. An ap
|
||||
## Requirements
|
||||
|
||||
- Go 1.27.
|
||||
- PostgreSQL 16 for any application that uses the data layer ([lagoon](modules/lagoon/README.md)). lagoon currently requires the database's default locale to be ICU `pl-PL`; see [Known issues](#known-issues).
|
||||
- PostgreSQL 16 for any application that uses the data layer ([lagoon](modules/lagoon/README.md)).
|
||||
- Node.js 22.6 or newer, only when working on the admin SPA in `admin/`.
|
||||
- Docker, only for the integration tests that start PostgreSQL or Mailpit containers through testcontainers-go.
|
||||
|
||||
@@ -32,10 +32,10 @@ summer build # generates plugins.gen.go and main.go, writes bin/hello
|
||||
|
||||
The `migrate`, `migrate:status`, `migrate:rollback`, `serve` and admin commands open PostgreSQL, so they need more configuration. Configuration comes from YAML files in `config/`, and any key can be overridden with a `SUMMER_`-prefixed environment variable in which a double underscore separates path segments (see [compass](modules/compass/README.md)):
|
||||
|
||||
1. Create a PostgreSQL 16 database with the locale lagoon checks for:
|
||||
1. Create a PostgreSQL 16 database:
|
||||
|
||||
```sql
|
||||
CREATE DATABASE hello TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL';
|
||||
CREATE DATABASE hello;
|
||||
```
|
||||
|
||||
2. Add the request body limits that `serve` and `route:list` require. The example does not ship them yet, and surf needs numeric values, which the string-valued environment overlay cannot supply. Create `config/http.yaml` in `examples/hello`:
|
||||
@@ -64,7 +64,6 @@ The `migrate`, `migrate:status`, `migrate:rollback`, `serve` and admin commands
|
||||
### Known issues
|
||||
|
||||
- `examples/hello` has no `http.body_limits` configuration, so `serve` and `route:list` fail with `surf: config http.body_limits.default_bytes is required` until you add it as in step 2. The same gap makes the example's `TestTypedItemRoute` fail.
|
||||
- lagoon refuses any database whose default locale is not ICU `pl-PL`.
|
||||
- The committed `examples/hello/main.go` is older than what `summer build` generates now, so building the example (or running its tests, which build it) leaves that file modified. Restore it with `git checkout -- examples/hello/main.go` if you do not intend to commit it.
|
||||
|
||||
## Repository layout
|
||||
|
||||
@@ -8,7 +8,7 @@ order: 10
|
||||
|
||||
A WinterCMS model extends Eloquent and describes its behaviour with properties such as `$fillable`, `$hidden` and `$jsonable`. A SummerCMS model is a plain GORM struct in the plugin's `models` package, and [lagoon](../../modules/lagoon/README.md) supplies the Eloquent conventions that GORM does not have: allow-listed mass assignment, JSON and encrypted columns, Laravel-style validation and pagination.
|
||||
|
||||
The data layer supports PostgreSQL only. `lagoon.Open` refuses a database whose default collation is not the ICU `pl-PL` locale (`lagoon.CheckLocale`), so text ordering is the same in every query without a `COLLATE` clause. Create the database with that locale, as shown in [Installation](../setup/installation.md).
|
||||
The data layer supports PostgreSQL only and puts no requirement on the database's default locale. A list that must sort text in one language's order passes a collation to `lagoon.OrderBy`, as described in [Queries and pagination](queries-and-pagination.md#sorting-with-a-collation).
|
||||
|
||||
## Defining a model
|
||||
|
||||
|
||||
@@ -39,7 +39,32 @@ fmt.Println(err)
|
||||
|
||||
Answer the error as a validation failure (422). The column must match an allow-list entry exactly, so list the qualified name (`acme_blog_posts.title`) when the query joins another table.
|
||||
|
||||
`lagoon.OrderBy` never adds a `COLLATE` clause. Text sorts by the database's ICU `pl-PL` default collation, which lagoon checks when it connects.
|
||||
## Sorting with a collation
|
||||
|
||||
Text sorts by the database's default collation unless you pass `lagoon.Collate`. A list that must follow one language's alphabet (Polish puts Ł between L and M, for example) passes `lagoon.Collate("pl-x-icu")`, and `lagoon.OrderBy` adds a `COLLATE` clause for that column only. ICU collations named `<language>-x-icu` exist in any PostgreSQL built with ICU support, which includes the official Docker images, so the database needs no special locale:
|
||||
|
||||
```go src=modules/lagoon/example_test.go#ExampleCollate
|
||||
// A dry-run handle shows the SQL without a database.
|
||||
db, _ := gorm.Open(postgres.New(postgres.Config{DSN: "host=127.0.0.1"}), &gorm.Config{DryRun: true, DisableAutomaticPing: true})
|
||||
allowed := []string{"title", "views"}
|
||||
|
||||
// Polish alphabetical order, whatever the database's default locale.
|
||||
q, err := lagoon.OrderBy(db.Model(&Post{}), "title", "asc", allowed, lagoon.Collate("pl-x-icu"))
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
var posts []Post
|
||||
fmt.Println(q.Find(&posts).Statement.SQL.String())
|
||||
|
||||
_, err = lagoon.OrderBy(db, "title", "asc", allowed, lagoon.Collate(`pl-x-icu" ASC, (SELECT 1) --`))
|
||||
fmt.Println(err)
|
||||
// Output:
|
||||
// SELECT * FROM "acme_blog_posts" WHERE "acme_blog_posts"."deleted_at" IS NULL ORDER BY title COLLATE "pl-x-icu" ASC
|
||||
// lagoon: order collation "pl-x-icu\" ASC, (SELECT 1) --" is not a valid collation name
|
||||
```
|
||||
|
||||
The collation name is validated (ASCII letters, digits, `_`, `-`, `.` and `@`, at most 63 bytes) and quoted as an identifier; anything else is an error, and no SQL is built. An index only helps that ORDER BY when it is built with the same collation.
|
||||
|
||||
## Pagination
|
||||
|
||||
|
||||
@@ -100,10 +100,9 @@ func stopPostgres() {
|
||||
}
|
||||
}
|
||||
|
||||
// icuDatabase creates a database for one test with the ICU pl-PL locale
|
||||
// lagoon requires, and drops it when the test ends. It returns the pool
|
||||
// opened on it and its DSN.
|
||||
func icuDatabase(t *testing.T) (*sql.DB, string) {
|
||||
// testDatabase creates a database for one test and drops it when the test
|
||||
// ends. It returns the pool opened on it and its DSN.
|
||||
func testDatabase(t *testing.T) (*sql.DB, string) {
|
||||
t.Helper()
|
||||
if testing.Short() {
|
||||
t.Skip("requires testcontainers postgres")
|
||||
@@ -113,7 +112,7 @@ func icuDatabase(t *testing.T) (*sql.DB, string) {
|
||||
}
|
||||
name := "blog_" + strings.ToLower(strings.NewReplacer("/", "_", "-", "_").Replace(t.Name()))
|
||||
quoted := `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
|
||||
if _, err := pgAdmin.ExecContext(t.Context(), `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`); err != nil {
|
||||
if _, err := pgAdmin.ExecContext(t.Context(), `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8'`); err != nil {
|
||||
t.Fatalf("create %s: %v", name, err)
|
||||
}
|
||||
u, err := url.Parse(pgDSN)
|
||||
@@ -133,11 +132,11 @@ func icuDatabase(t *testing.T) (*sql.DB, string) {
|
||||
return db, dsn
|
||||
}
|
||||
|
||||
// migrated activates acme.blog, migrates a fresh ICU database and publishes
|
||||
// migrated activates acme.blog, migrates a fresh database and publishes
|
||||
// it on the application, as the serve command does at start-up.
|
||||
func migrated(t *testing.T) (*backpack.App, party.Plugin, *gorm.DB) {
|
||||
t.Helper()
|
||||
sqlDB, _ := icuDatabase(t)
|
||||
sqlDB, _ := testDatabase(t)
|
||||
gdb, err := lagoon.Use(t.Context(), sqlDB)
|
||||
if err != nil {
|
||||
t.Fatalf("lagoon.Use: %v", err)
|
||||
|
||||
@@ -31,10 +31,9 @@ Most plugin code runs without a database. Build a container with `backpack.New`,
|
||||
|
||||
## Database tests
|
||||
|
||||
SummerCMS supports PostgreSQL only, so database tests run against real PostgreSQL rather than an SQLite stand-in. The framework's own tests start a `postgres:16-alpine` container through testcontainers-go and create the database with the ICU `pl-PL` locale that lagoon checks for when it connects. Follow the same pattern in plugin tests:
|
||||
SummerCMS supports PostgreSQL only, so database tests run against real PostgreSQL rather than an SQLite stand-in. The framework's own tests start a `postgres:16-alpine` container through testcontainers-go and create a fresh database per test. Follow the same pattern in plugin tests:
|
||||
|
||||
- skip the test when `testing.Short` reports true;
|
||||
- create the database with `LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`, or lagoon refuses the connection;
|
||||
- migrate a fresh database per test with `lagoon.Migrate`, so tests do not depend on each other.
|
||||
|
||||
Docker must be running for these tests.
|
||||
|
||||
@@ -11,7 +11,7 @@ SummerCMS is a Go module. An application requires it, lists its plugins in a `su
|
||||
## Requirements
|
||||
|
||||
- Go 1.27.
|
||||
- PostgreSQL 16 for any application that uses the data layer. The database's default locale must be the ICU `pl-PL` locale, which lagoon checks when it connects.
|
||||
- PostgreSQL 16 for any application that uses the data layer.
|
||||
- Docker, only for the integration tests that start PostgreSQL or Mailpit containers.
|
||||
|
||||
You do not need Node.js to build an application or these docs. It is needed only when you work on the admin SPA itself.
|
||||
@@ -74,10 +74,10 @@ summer build
|
||||
|
||||
## Create the database
|
||||
|
||||
The commands that touch data open PostgreSQL. Create a database with the locale lagoon checks for:
|
||||
The commands that touch data open PostgreSQL. Create a database for the example:
|
||||
|
||||
```sql
|
||||
CREATE DATABASE hello TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL';
|
||||
CREATE DATABASE hello;
|
||||
```
|
||||
|
||||
## Configure the application
|
||||
@@ -117,7 +117,6 @@ From the application directory, `summer migrate`, `summer migrate:status` and `s
|
||||
> Known issues in the current framework:
|
||||
>
|
||||
> - `examples/hello` ships no `http.body_limits` configuration, so `serve` and `route:list` fail with `surf: config http.body_limits.default_bytes is required` until you add `config/http.yaml` as shown above. The same gap makes the example's `TestTypedItemRoute` fail.
|
||||
> - lagoon refuses any database whose default locale is not ICU `pl-PL`.
|
||||
> - The committed `examples/hello/main.go` is older than what `summer build` generates now, so building the example leaves that file modified. Restore it with `git checkout -- examples/hello/main.go` if you do not intend to commit it.
|
||||
|
||||
## Next steps
|
||||
|
||||
@@ -58,9 +58,6 @@ func startBeachcomberPostgres(ctx context.Context) error {
|
||||
postgres.WithUsername("beachcomber"),
|
||||
postgres.WithPassword("beachcomber"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
testcontainers.WithEnv(map[string]string{
|
||||
"POSTGRES_INITDB_ARGS": "--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8",
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -106,14 +103,14 @@ func adminDB(t *testing.T) *sql.DB {
|
||||
return bcSQL
|
||||
}
|
||||
|
||||
// migratedDB returns a dedicated ICU pl-PL database migrated with
|
||||
// migratedDB returns a dedicated database migrated with
|
||||
// lagoon.Migrate (River v7 and summer_jobs) plus its DSN.
|
||||
func migratedDB(t *testing.T) (*sql.DB, string) {
|
||||
t.Helper()
|
||||
admin := adminDB(t)
|
||||
ctx := t.Context()
|
||||
name := fmt.Sprintf("beachcomber_%d", dbSeq.Add(1))
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+name+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+name+` TEMPLATE template0 ENCODING 'UTF8'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
t.Fatalf("create %s: %v", name, err)
|
||||
}
|
||||
dsn, err := dsnWithDB(bcDSN, name)
|
||||
|
||||
@@ -8,7 +8,6 @@ import (
|
||||
"time"
|
||||
|
||||
_ "github.com/jackc/pgx/v5/stdlib"
|
||||
"github.com/testcontainers/testcontainers-go"
|
||||
"github.com/testcontainers/testcontainers-go/modules/postgres"
|
||||
)
|
||||
|
||||
@@ -70,9 +69,6 @@ func TestPostgresBlacklistConcurrent(t *testing.T) {
|
||||
postgres.WithUsername("bl"),
|
||||
postgres.WithPassword("bl"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
testcontainers.WithEnv(map[string]string{
|
||||
"POSTGRES_INITDB_ARGS": "--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8",
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
|
||||
@@ -28,7 +28,6 @@ import (
|
||||
"git.golem15.com/golem15/summercms/modules/party"
|
||||
"git.golem15.com/golem15/summercms/modules/surf"
|
||||
_ "github.com/jackc/pgx/v5/stdlib"
|
||||
"github.com/testcontainers/testcontainers-go"
|
||||
"github.com/testcontainers/testcontainers-go/modules/postgres"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
@@ -332,9 +331,6 @@ func adminGorm(t *testing.T) *gorm.DB {
|
||||
postgres.WithUsername("cabana"),
|
||||
postgres.WithPassword("cabana"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
testcontainers.WithEnv(map[string]string{
|
||||
"POSTGRES_INITDB_ARGS": "--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8",
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
adminErr = err
|
||||
|
||||
@@ -61,9 +61,6 @@ func startCongaPostgres(ctx context.Context) error {
|
||||
postgres.WithUsername("conga"),
|
||||
postgres.WithPassword("conga"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
testcontainers.WithEnv(map[string]string{
|
||||
"POSTGRES_INITDB_ARGS": "--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8",
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -109,14 +106,14 @@ func adminDB(t *testing.T) *sql.DB {
|
||||
return congaSQL
|
||||
}
|
||||
|
||||
// migratedDB returns a dedicated ICU pl-PL database migrated with
|
||||
// migratedDB returns a dedicated database migrated with
|
||||
// lagoon.Migrate (River v7 and summer_jobs) plus its DSN.
|
||||
func migratedDB(t *testing.T) (*sql.DB, string) {
|
||||
t.Helper()
|
||||
admin := adminDB(t)
|
||||
ctx := t.Context()
|
||||
name := fmt.Sprintf("conga_%d", dbSeq.Add(1))
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+name+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+name+` TEMPLATE template0 ENCODING 'UTF8'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
t.Fatalf("create %s: %v", name, err)
|
||||
}
|
||||
dsn, err := dsnWithDB(congaDSN, name)
|
||||
|
||||
@@ -15,11 +15,10 @@ Postgres data layer: the shared GORM connection, per-plugin migrations, model he
|
||||
- One shared pool: `lagoon.Open`, `lagoon.Use` and `lagoon.OpenFromApp` return a `*sql.DB` and a `*gorm.DB` built on that same pool; `lagoon.Publish` makes both available on the `backpack.App`.
|
||||
- Database-ready hooks: `lagoon.OnDatabase` runs a callback with the pool and GORM handle as soon as the database is published, immediately when it already is, otherwise when `lagoon.Publish` runs. Plugins register GORM callbacks through it from Boot, which runs before the `serve` command publishes the database.
|
||||
- After-commit work: `lagoon.Transaction` runs a function in a transaction and then the callbacks registered with `lagoon.AfterCommit`, in order, only after the commit succeeds; a nested `lagoon.Transaction` is a savepoint whose callbacks are dropped with it when it fails. A nested `lagoon.Transaction` must be given the outer transaction's handle: given a root handle it returns an error without running its function, rather than open an independent transaction whose callbacks would wait on the outer one. A single-statement write for which GORM opens its own implicit transaction runs its callbacks from `lagoon:after_commit` once GORM commits, and never when the write fails. A callback registered inside a foreign plain GORM transaction is unsafe because Lagoon cannot observe its commit, so `lagoon.AfterCommit` warns and skips it. Outside a transaction, callbacks run immediately. The handle a supported callback receives always has an empty statement on the connection its work belongs to. A panicking callback is logged and never turns a committed write into an error.
|
||||
- Database check at connect time: `lagoon.CheckLocale` refuses a database whose default collation is not the ICU `pl-PL` locale, so ordering matches the database default without per-query `COLLATE`.
|
||||
- Per-plugin migrations: `lagoon.Migrate` runs the framework's `system_files` set (`attach.Migrations`), backend admin identity set (`lagoon.BackendAdminMigrations`) and job-queue set (`lagoon.QueueMigrations`: River's schema pinned at `lagoon.RiverSchemaVersion`, then the `lagoon.JobsTable` record table, under the `lagoon.QueueHistoryID` history), then every `pact.HasMigrations` set in plugin activation order, each in its own `summer_migrations_<plugin_id>` history table (`lagoon.HistoryTableName`). `lagoon.RollbackLast` and `lagoon.Status` cover rollback and history.
|
||||
- Mass assignment: `lagoon.Fill` copies only allow-listed keys onto a model by GORM column name and silently drops the rest, logging each dropped key once outside production. A `json.Number` (from a decoder using `UseNumber`) fills integer, unsigned and float fields. A value that does not fit its column (a fraction, an exponent or an overflow for an integer field, or a value of the wrong type) is a `lagoon.FillTypeError` naming the key, so a caller can answer it as a validation failure on that field. `lagoon.HasFillable` and `lagoon.HasHidden` are the Go forms of `$fillable` and `$hidden`.
|
||||
- Validation: `lagoon.Validate` accepts Laravel-style rule strings (`required`, `nullable`, `integer`, `numeric`, `between`, `min`, `max`, `in`, `unique`, `boolean`, `email`, `confirmed`, `different`, `mimes`) and returns a field-to-messages map, translated through phrasebook when a translator is given. Unknown rule tokens are an error.
|
||||
- Safe ordering: `lagoon.OrderBy` appends an ORDER BY only for an allow-listed column and an `asc` or `desc` direction.
|
||||
- Safe ordering: `lagoon.OrderBy` appends an ORDER BY only for an allow-listed column and an `asc` or `desc` direction, and `lagoon.Collate` adds a validated `COLLATE` clause for language-specific text order (for example the ICU collation `pl-x-icu`); lagoon puts no requirement on the database's default locale.
|
||||
- Pagination: `lagoon.Paginate` builds a `lagoon.Page` with `data` and `meta` (`current_page`, `last_page`, `per_page`, `total`).
|
||||
- Column types: `lagoon.Encrypted` stores AES-256-GCM ciphertext under a key derived from `app.key`, decrypts with previous keys during rotation, and always redacts itself in JSON and string output; `lagoon.Jsonable` stores JSON as TEXT and keeps SQL NULL distinct from an empty value.
|
||||
- Lifecycle and relations: hook interfaces matching GORM's native method names (`lagoon.HasBeforeCreate`, `lagoon.HasBeforeSave`, `lagoon.HasBeforeDelete`, `lagoon.HasAfterDelete`) plus `lagoon.HasBeforeValidate`; `lagoon.WithSoftDeleteCascade` runs a cascade inside the parent delete; `lagoon.RegisterJoinTable` wires pivot models with business columns.
|
||||
@@ -110,15 +109,14 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
|
||||
| Identifier | Description |
|
||||
|------------|-------------|
|
||||
| `lagoon.OpenFromApp` | Opens the shared pool from `database.dsn` and publishes the `app.key` encryption keys. |
|
||||
| `lagoon.Open` | Opens and pings a DSN, checks the locale and returns the pool plus a GORM handle on it. |
|
||||
| `lagoon.Use` | Returns a GORM handle on an existing pool after the same checks. |
|
||||
| `lagoon.Open` | Opens and pings a DSN and returns the pool plus a GORM handle on it. |
|
||||
| `lagoon.Use` | Returns a GORM handle on an existing pool after a ping. |
|
||||
| `lagoon.Publish` | Stores the pool and GORM handle on the `backpack.App`, then runs the callbacks queued by `lagoon.OnDatabase`. |
|
||||
| `lagoon.OnDatabase` | Runs a callback with the pool and GORM handle once the database is published. |
|
||||
| `lagoon.Transaction` | Runs a function in a transaction (a savepoint when nested) and its `lagoon.AfterCommit` callbacks after the commit. |
|
||||
| `lagoon.AfterCommit` | Registers work to run after the surrounding transaction commits. |
|
||||
| `lagoon.AfterCommitCallback` | Name of the GORM callback, `lagoon:after_commit`, that runs single-statement after-commit work. |
|
||||
| `lagoon.DSN` | Reads `database.dsn` from config. |
|
||||
| `lagoon.CheckLocale` | Fails unless the database default locale is ICU `pl-PL`. |
|
||||
| `lagoon.Migrate` | Runs framework and plugin migrations in order. |
|
||||
| `lagoon.RollbackLast` | Rolls back the last migration of one plugin. |
|
||||
| `lagoon.Status` | Lists applied migration IDs per plugin as `lagoon.StatusRow` values. |
|
||||
@@ -137,7 +135,9 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
|
||||
| `lagoon.Fill` | Allow-listed mass assignment by column name. |
|
||||
| `lagoon.FillTypeError` | Returned by `lagoon.Fill` when a requested value does not fit its column; `Key` names the column. |
|
||||
| `lagoon.Validate` | Laravel-style rule validation with a `unique` database check. |
|
||||
| `lagoon.OrderBy` | Allow-listed ORDER BY. |
|
||||
| `lagoon.OrderBy` | Allow-listed ORDER BY, with an optional `lagoon.Collate`. |
|
||||
| `lagoon.Collate` | Order option that sorts the column with a named PostgreSQL collation; the name is validated and quoted. |
|
||||
| `lagoon.OrderOption` | Option type accepted by `lagoon.OrderBy`. |
|
||||
| `lagoon.Paginate` | Builds a `lagoon.Page` with `lagoon.PageMeta`. |
|
||||
| `lagoon.WithSoftDeleteCascade` | Runs a cascade inside the parent delete transaction. |
|
||||
| `lagoon.RegisterJoinTable` | Registers a custom pivot model for a many-to-many field. |
|
||||
@@ -198,4 +198,4 @@ storage:
|
||||
go test ./modules/lagoon/...
|
||||
```
|
||||
|
||||
The database tests in `lagoon` and `lagoon/attach` start a `postgres:16-alpine` container, initialised with the ICU `pl-PL` locale, through testcontainers-go, so they need a running Docker daemon. They are skipped by `go test -short ./modules/lagoon/...`, which runs only the unit tests.
|
||||
The database tests in `lagoon` and `lagoon/attach` start a `postgres:16-alpine` container through testcontainers-go, so they need a running Docker daemon. They are skipped by `go test -short ./modules/lagoon/...`, which runs only the unit tests.
|
||||
|
||||
@@ -275,9 +275,6 @@ func attachGorm(t *testing.T) *gorm.DB {
|
||||
postgres.WithUsername("attach"),
|
||||
postgres.WithPassword("attach"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
testcontainers.WithEnv(map[string]string{
|
||||
"POSTGRES_INITDB_ARGS": "--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8",
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("postgres: %v", err)
|
||||
|
||||
@@ -13,13 +13,8 @@ import (
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
const (
|
||||
requiredLocaleProvider = "i"
|
||||
requiredICULocale = "pl-PL"
|
||||
)
|
||||
|
||||
// Open pings dsn through pgx stdlib, requires Postgres 16 ICU pl-PL, and
|
||||
// returns that exact *sql.DB plus a GORM handle opened on it.
|
||||
// Open pings dsn through pgx stdlib and returns that exact *sql.DB plus a
|
||||
// GORM handle opened on it.
|
||||
//
|
||||
// The job worker in the conga module opens its own single-connection
|
||||
// pgxpool.Pool for River LISTEN/NOTIFY. Do not create that listener pool
|
||||
@@ -37,10 +32,6 @@ func Open(ctx context.Context, dsn string) (*sql.DB, *gorm.DB, error) {
|
||||
_ = sqlDB.Close()
|
||||
return nil, nil, fmt.Errorf("lagoon: ping postgres: %w", err)
|
||||
}
|
||||
if err := CheckLocale(ctx, sqlDB); err != nil {
|
||||
_ = sqlDB.Close()
|
||||
return nil, nil, err
|
||||
}
|
||||
gdb, err := gormFromSQL(sqlDB)
|
||||
if err != nil {
|
||||
_ = sqlDB.Close()
|
||||
@@ -49,9 +40,9 @@ func Open(ctx context.Context, dsn string) (*sql.DB, *gorm.DB, error) {
|
||||
return sqlDB, gdb, nil
|
||||
}
|
||||
|
||||
// Use pings an existing pool, requires ICU pl-PL, and returns a GORM handle
|
||||
// opened on that exact *sql.DB. Callers that already hold a pool (tests,
|
||||
// the app boot seam) must not open a second connection.
|
||||
// Use pings an existing pool and returns a GORM handle opened on that exact
|
||||
// *sql.DB. Callers that already hold a pool (tests, the app boot seam) must
|
||||
// not open a second connection.
|
||||
func Use(ctx context.Context, sqlDB *sql.DB) (*gorm.DB, error) {
|
||||
if sqlDB == nil {
|
||||
return nil, fmt.Errorf("lagoon: sql db is nil")
|
||||
@@ -59,9 +50,6 @@ func Use(ctx context.Context, sqlDB *sql.DB) (*gorm.DB, error) {
|
||||
if err := sqlDB.PingContext(ctx); err != nil {
|
||||
return nil, fmt.Errorf("lagoon: ping postgres: %w", err)
|
||||
}
|
||||
if err := CheckLocale(ctx, sqlDB); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gormFromSQL(sqlDB)
|
||||
}
|
||||
|
||||
@@ -127,28 +115,3 @@ func Publish(app *backpack.App, sqlDB *sql.DB, gdb *gorm.DB) error {
|
||||
}
|
||||
return runDatabaseHooks(app, sqlDB, gdb)
|
||||
}
|
||||
|
||||
// CheckLocale fails unless the connected database uses ICU locale pl-PL.
|
||||
func CheckLocale(ctx context.Context, db *sql.DB) error {
|
||||
if db == nil {
|
||||
return fmt.Errorf("lagoon: sql db is nil")
|
||||
}
|
||||
var provider, icu string
|
||||
err := db.QueryRowContext(ctx, `
|
||||
SELECT datlocprovider::text, COALESCE(daticulocale, '')
|
||||
FROM pg_database
|
||||
WHERE datname = current_database()`).Scan(&provider, &icu)
|
||||
if err != nil {
|
||||
return fmt.Errorf("lagoon: read database locale: %w", err)
|
||||
}
|
||||
return checkLocale(provider, icu)
|
||||
}
|
||||
|
||||
func checkLocale(provider, icu string) error {
|
||||
provider = strings.TrimSpace(provider)
|
||||
icu = strings.TrimSpace(icu)
|
||||
if provider == requiredLocaleProvider && icu == requiredICULocale {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("lagoon: database locale must be ICU pl-PL (datlocprovider=%q, daticulocale=%q); got provider %q locale %q. Create the database with: CREATE DATABASE ... TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'", requiredLocaleProvider, requiredICULocale, provider, icu)
|
||||
}
|
||||
|
||||
@@ -21,13 +21,6 @@ func TestUseRejectsNilSQL(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckLocaleRejectsNilSQL(t *testing.T) {
|
||||
err := CheckLocale(t.Context(), nil)
|
||||
if err == nil || !strings.Contains(err.Error(), "sql db is nil") {
|
||||
t.Fatalf("got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPublishRejectsNilHandles(t *testing.T) {
|
||||
app := backpack.New(nil)
|
||||
if err := Publish(nil, nil, nil); err == nil {
|
||||
@@ -105,31 +98,6 @@ func TestSharedSQLPoolUsedByGORMAndClosed(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrongICULocaleFailsOpen(t *testing.T) {
|
||||
admin := lagoonDB(t)
|
||||
ctx := t.Context()
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE lagoon_locale_fail TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER libc LOCALE 'C'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
t.Fatalf("create libc database: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_, _ = admin.ExecContext(ctx, `DROP DATABASE IF EXISTS lagoon_locale_fail WITH (FORCE)`)
|
||||
})
|
||||
failDSN, err := dsnWithDB(lagoonDSN, "lagoon_locale_fail")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, _, err = Open(ctx, failDSN)
|
||||
if err == nil {
|
||||
t.Fatal("wrong locale must fail boot")
|
||||
}
|
||||
msg := err.Error()
|
||||
for _, want := range []string{"ICU", "pl-PL", "CREATE DATABASE", "LOCALE_PROVIDER icu"} {
|
||||
if !strings.Contains(msg, want) {
|
||||
t.Fatalf("missing %q in %s", want, msg)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenFromAppReadsDSN(t *testing.T) {
|
||||
_, dsn := dedicatedDB(t, "lagoon_from_app")
|
||||
dir := t.TempDir()
|
||||
|
||||
@@ -208,6 +208,27 @@ func ExampleOrderBy() {
|
||||
// lagoon: order direction "asc; DROP TABLE acme_blog_posts" is not allow-listed
|
||||
}
|
||||
|
||||
func ExampleCollate() {
|
||||
// A dry-run handle shows the SQL without a database.
|
||||
db, _ := gorm.Open(postgres.New(postgres.Config{DSN: "host=127.0.0.1"}), &gorm.Config{DryRun: true, DisableAutomaticPing: true})
|
||||
allowed := []string{"title", "views"}
|
||||
|
||||
// Polish alphabetical order, whatever the database's default locale.
|
||||
q, err := lagoon.OrderBy(db.Model(&Post{}), "title", "asc", allowed, lagoon.Collate("pl-x-icu"))
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
var posts []Post
|
||||
fmt.Println(q.Find(&posts).Statement.SQL.String())
|
||||
|
||||
_, err = lagoon.OrderBy(db, "title", "asc", allowed, lagoon.Collate(`pl-x-icu" ASC, (SELECT 1) --`))
|
||||
fmt.Println(err)
|
||||
// Output:
|
||||
// SELECT * FROM "acme_blog_posts" WHERE "acme_blog_posts"."deleted_at" IS NULL ORDER BY title COLLATE "pl-x-icu" ASC
|
||||
// lagoon: order collation "pl-x-icu\" ASC, (SELECT 1) --" is not a valid collation name
|
||||
}
|
||||
|
||||
func ExamplePaginate() {
|
||||
rows := []map[string]any{{"id": 3, "title": "Third"}}
|
||||
page := lagoon.Paginate(rows, 2, 2, 3)
|
||||
|
||||
@@ -6,8 +6,8 @@ import (
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// DocsDB returns a GORM handle on a fresh ICU pl-PL database named name in
|
||||
// this package's Postgres harness, for the database-backed docs examples in
|
||||
// DocsDB returns a GORM handle on a fresh database named name in this
|
||||
// package's Postgres harness, for the database-backed docs examples in
|
||||
// example_test.go. It skips under -short and fails when the harness has no
|
||||
// database, like the package's other database tests.
|
||||
func DocsDB(t *testing.T, name string) *gorm.DB {
|
||||
|
||||
@@ -39,22 +39,6 @@ func TestHistoryTableName(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckLocaleMessage(t *testing.T) {
|
||||
if err := checkLocale("i", "pl-PL"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := checkLocale("c", "en_US")
|
||||
if err == nil {
|
||||
t.Fatal("want locale error")
|
||||
}
|
||||
msg := err.Error()
|
||||
for _, want := range []string{"ICU", "pl-PL", "CREATE DATABASE", "LOCALE_PROVIDER icu", "en_US"} {
|
||||
if !strings.Contains(msg, want) {
|
||||
t.Fatalf("missing %q in %s", want, msg)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRuntimeCommandsRegisterBareAndColonNames(t *testing.T) {
|
||||
cmds := RuntimeCommands(nil, nil)
|
||||
names := map[string]bool{}
|
||||
|
||||
@@ -7,33 +7,100 @@ import (
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// OrderBy appends a database-default ORDER BY for an allow-listed qualified
|
||||
// column. Identifiers and directions are never taken from untrusted input:
|
||||
// column must match allowed exactly, and dir must be asc or desc. No COLLATE
|
||||
// is emitted; Postgres ICU pl-PL is the database default (CheckLocale).
|
||||
func OrderBy(db *gorm.DB, column, dir string, allowed []string) (*gorm.DB, error) {
|
||||
// maxCollationName is PostgreSQL's identifier limit (NAMEDATALEN-1).
|
||||
const maxCollationName = 63
|
||||
|
||||
// OrderOption changes how OrderBy builds its ORDER BY clause.
|
||||
type OrderOption func(*orderOptions)
|
||||
|
||||
type orderOptions struct {
|
||||
collation string
|
||||
hasCollation bool
|
||||
}
|
||||
|
||||
// Collate sorts the column with the named PostgreSQL collation, for example
|
||||
// the ICU collation pl-x-icu for Polish alphabetical order. ICU collations
|
||||
// exist in any ICU-enabled PostgreSQL server, so they need no database
|
||||
// setup. The name is validated and emitted as a double-quoted identifier;
|
||||
// an invalid name makes OrderBy return an error.
|
||||
func Collate(name string) OrderOption {
|
||||
return func(o *orderOptions) {
|
||||
o.collation = name
|
||||
o.hasCollation = true
|
||||
}
|
||||
}
|
||||
|
||||
// OrderBy appends an ORDER BY for an allow-listed qualified column.
|
||||
// Identifiers and directions are never taken from untrusted input: column
|
||||
// must match allowed exactly, and dir must be asc or desc. No COLLATE is
|
||||
// emitted unless Collate is passed; without it, text sorts by the
|
||||
// database's default collation.
|
||||
func OrderBy(db *gorm.DB, column, dir string, allowed []string, opts ...OrderOption) (*gorm.DB, error) {
|
||||
if db == nil {
|
||||
return nil, fmt.Errorf("lagoon: gorm db is nil")
|
||||
}
|
||||
clause, err := orderClause(column, dir, allowed)
|
||||
clause, err := orderClause(column, dir, allowed, resolveOrderOptions(opts))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return db.Order(clause), nil
|
||||
}
|
||||
|
||||
func orderClause(column, dir string, allowed []string) (string, error) {
|
||||
func resolveOrderOptions(opts []OrderOption) orderOptions {
|
||||
var o orderOptions
|
||||
for _, opt := range opts {
|
||||
if opt != nil {
|
||||
opt(&o)
|
||||
}
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
func orderClause(column, dir string, allowed []string, opts orderOptions) (string, error) {
|
||||
if !allowListed(column, allowed) {
|
||||
return "", fmt.Errorf("lagoon: order column %q is not allow-listed", column)
|
||||
}
|
||||
var direction string
|
||||
switch strings.ToLower(strings.TrimSpace(dir)) {
|
||||
case "asc":
|
||||
return column + " ASC", nil
|
||||
direction = "ASC"
|
||||
case "desc":
|
||||
return column + " DESC", nil
|
||||
direction = "DESC"
|
||||
default:
|
||||
return "", fmt.Errorf("lagoon: order direction %q is not allow-listed", dir)
|
||||
}
|
||||
if !opts.hasCollation {
|
||||
return column + " " + direction, nil
|
||||
}
|
||||
if !validCollationName(opts.collation) {
|
||||
return "", fmt.Errorf("lagoon: order collation %q is not a valid collation name", opts.collation)
|
||||
}
|
||||
return column + " COLLATE " + quoteCollation(opts.collation) + " " + direction, nil
|
||||
}
|
||||
|
||||
// validCollationName accepts 1-63 ASCII bytes: letters, digits and
|
||||
// underscore first, then letters, digits, '_', '-', '.' and '@'. The set
|
||||
// excludes the double quote, so a valid name cannot leave its identifier.
|
||||
func validCollationName(name string) bool {
|
||||
if len(name) == 0 || len(name) > maxCollationName {
|
||||
return false
|
||||
}
|
||||
for i := 0; i < len(name); i++ {
|
||||
c := name[i]
|
||||
switch {
|
||||
case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == '_':
|
||||
case i > 0 && (c == '-' || c == '.' || c == '@'):
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// quoteCollation wraps a validated collation name as a PostgreSQL quoted
|
||||
// identifier.
|
||||
func quoteCollation(name string) string {
|
||||
return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
|
||||
}
|
||||
|
||||
func allowListed(column string, allowed []string) bool {
|
||||
|
||||
@@ -1,42 +1,219 @@
|
||||
package lagoon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestOrderClauseAllowList(t *testing.T) {
|
||||
allowed := []string{"golem15_fonoteka_genres.name", "items.title"}
|
||||
allowed := []string{"acme_blog_posts.title", "items.title"}
|
||||
|
||||
got, err := orderClause("golem15_fonoteka_genres.name", "asc", allowed)
|
||||
got, err := orderClause("acme_blog_posts.title", "asc", allowed, orderOptions{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "golem15_fonoteka_genres.name ASC" {
|
||||
if got != "acme_blog_posts.title ASC" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
if strings.Contains(strings.ToLower(got), "collate") {
|
||||
t.Fatalf("must not emit COLLATE: %q", got)
|
||||
}
|
||||
|
||||
got, err = orderClause("items.title", "DESC", allowed)
|
||||
got, err = orderClause("items.title", "DESC", allowed, orderOptions{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "items.title DESC" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
if strings.Contains(strings.ToLower(got), "collate") {
|
||||
t.Fatalf("must not emit COLLATE: %q", got)
|
||||
}
|
||||
|
||||
if _, err := orderClause("golem15_fonoteka_genres.name;drop table x", "asc", allowed); err == nil {
|
||||
if _, err := orderClause("acme_blog_posts.title;drop table x", "asc", allowed, orderOptions{}); err == nil {
|
||||
t.Fatal("want reject unknown column")
|
||||
}
|
||||
if _, err := orderClause("golem15_fonoteka_genres.name", "ascending", allowed); err == nil {
|
||||
if _, err := orderClause("acme_blog_posts.title", "ascending", allowed, orderOptions{}); err == nil {
|
||||
t.Fatal("want reject unknown direction")
|
||||
}
|
||||
if _, err := OrderBy(nil, "items.title", "asc", allowed); err == nil {
|
||||
t.Fatal("want nil db error")
|
||||
}
|
||||
if _, err := orderClause("items.title", "asc", nil); err == nil {
|
||||
if _, err := orderClause("items.title", "asc", nil, orderOptions{}); err == nil {
|
||||
t.Fatal("empty allow-list must reject")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrderClauseCollation(t *testing.T) {
|
||||
allowed := []string{"items.title"}
|
||||
opts := resolveOrderOptions([]OrderOption{Collate("pl-x-icu")})
|
||||
|
||||
got, err := orderClause("items.title", "asc", allowed, opts)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != `items.title COLLATE "pl-x-icu" ASC` {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
got, err = orderClause("items.title", "DESC", allowed, opts)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != `items.title COLLATE "pl-x-icu" DESC` {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollateAcceptsValidNames(t *testing.T) {
|
||||
allowed := []string{"items.title"}
|
||||
for _, name := range []string{
|
||||
"pl-x-icu",
|
||||
"und-x-icu",
|
||||
"en-US-x-icu",
|
||||
"C",
|
||||
"POSIX",
|
||||
"ucs_basic",
|
||||
"default",
|
||||
"sr_RS.utf8@latin",
|
||||
strings.Repeat("a", 63),
|
||||
} {
|
||||
got, err := orderClause("items.title", "asc", allowed, resolveOrderOptions([]OrderOption{Collate(name)}))
|
||||
if err != nil {
|
||||
t.Fatalf("%q: %v", name, err)
|
||||
}
|
||||
want := `items.title COLLATE "` + name + `" ASC`
|
||||
if got != want {
|
||||
t.Fatalf("%q: got %q want %q", name, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollateRejectsInvalidNames(t *testing.T) {
|
||||
allowed := []string{"items.title"}
|
||||
db := &gorm.DB{}
|
||||
for _, name := range []string{
|
||||
"",
|
||||
strings.Repeat("a", 64),
|
||||
"pl x icu",
|
||||
`pl-x-icu"`,
|
||||
`pl-x-icu" ASC, (SELECT 1) --`,
|
||||
"pl;DROP TABLE x",
|
||||
`pl\x-icu`,
|
||||
"pl\x00x-icu",
|
||||
"pł-x-icu",
|
||||
"-pl-x-icu",
|
||||
".pl-x-icu",
|
||||
} {
|
||||
want := fmt.Sprintf("lagoon: order collation %q is not a valid collation name", name)
|
||||
_, err := orderClause("items.title", "asc", allowed, resolveOrderOptions([]OrderOption{Collate(name)}))
|
||||
if err == nil || err.Error() != want {
|
||||
t.Fatalf("%q: got %v want %s", name, err, want)
|
||||
}
|
||||
q, err := OrderBy(db, "items.title", "asc", allowed, Collate(name))
|
||||
if q != nil || err == nil || err.Error() != want {
|
||||
t.Fatalf("OrderBy %q: got (%v, %v)", name, q, err)
|
||||
}
|
||||
}
|
||||
if db.Statement != nil {
|
||||
t.Fatal("rejected collation must not touch the GORM handle")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollateOptionOrdering(t *testing.T) {
|
||||
allowed := []string{"items.title"}
|
||||
|
||||
got, err := orderClause("items.title", "asc", allowed, resolveOrderOptions([]OrderOption{Collate("C"), Collate("pl-x-icu")}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != `items.title COLLATE "pl-x-icu" ASC` {
|
||||
t.Fatalf("last Collate must win: %q", got)
|
||||
}
|
||||
|
||||
got, err = orderClause("items.title", "asc", allowed, resolveOrderOptions(nil))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "items.title ASC" {
|
||||
t.Fatalf("no options must keep the plain clause: %q", got)
|
||||
}
|
||||
|
||||
// Column and direction are checked before the collation.
|
||||
bad := resolveOrderOptions([]OrderOption{Collate(`bad"name`)})
|
||||
_, err = orderClause("items.other", "asc", allowed, bad)
|
||||
if err == nil || !strings.Contains(err.Error(), "order column") {
|
||||
t.Fatalf("column check must run first: %v", err)
|
||||
}
|
||||
_, err = orderClause("items.title", "sideways", allowed, bad)
|
||||
if err == nil || !strings.Contains(err.Error(), "order direction") {
|
||||
t.Fatalf("direction check must run second: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollatePolishOrderOnLibcDatabase(t *testing.T) {
|
||||
admin := lagoonDB(t)
|
||||
ctx := t.Context()
|
||||
const name = "lagoon_collate_libc"
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+name+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER libc LOCALE 'C'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
t.Fatalf("create libc database: %v", err)
|
||||
}
|
||||
dsn, err := dsnWithDB(lagoonDSN, name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sqlDB, gdb, err := Open(ctx, dsn)
|
||||
if err != nil {
|
||||
t.Fatalf("Open must accept a libc 'C' database: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_ = sqlDB.Close()
|
||||
_, _ = admin.ExecContext(context.Background(), `DROP DATABASE IF EXISTS `+name+` WITH (FORCE)`)
|
||||
})
|
||||
if _, err := Use(ctx, sqlDB); err != nil {
|
||||
t.Fatalf("Use must accept a libc 'C' database: %v", err)
|
||||
}
|
||||
if _, err := sqlDB.ExecContext(ctx, `CREATE TABLE words (name text)`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := sqlDB.ExecContext(ctx, `INSERT INTO words (name) VALUES ('Zebra'), ('Łoś'), ('Lis'), ('Mysz')`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
allowed := []string{"words.name"}
|
||||
scan := func(opts ...OrderOption) ([]string, error) {
|
||||
q, err := OrderBy(gdb.Table("words"), "words.name", "asc", allowed, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
if err := q.Pluck("name", &names).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
plain, err := scan()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := []string{"Lis", "Mysz", "Zebra", "Łoś"}; !reflect.DeepEqual(plain, want) {
|
||||
t.Fatalf("plain order on libc 'C': got %v want %v", plain, want)
|
||||
}
|
||||
|
||||
polish, err := scan(Collate("pl-x-icu"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := []string{"Lis", "Łoś", "Mysz", "Zebra"}; !reflect.DeepEqual(polish, want) {
|
||||
t.Fatalf("pl-x-icu order: got %v want %v", polish, want)
|
||||
}
|
||||
|
||||
_, err = scan(Collate("no-such-collation-x"))
|
||||
if err == nil || !strings.Contains(err.Error(), "no-such-collation-x") {
|
||||
t.Fatalf("unknown collation must reach Postgres as an identifier: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -55,9 +55,6 @@ func startLagoonPostgres(ctx context.Context) error {
|
||||
postgres.WithUsername("lagoon"),
|
||||
postgres.WithPassword("lagoon"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
testcontainers.WithEnv(map[string]string{
|
||||
"POSTGRES_INITDB_ARGS": "--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8",
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -108,7 +105,7 @@ func dedicatedDB(t *testing.T, name string) (*sql.DB, string) {
|
||||
admin := lagoonDB(t)
|
||||
ctx := t.Context()
|
||||
quoted := `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+quoted+` TEMPLATE template0 ENCODING 'UTF8'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
t.Fatalf("create %s: %v", name, err)
|
||||
}
|
||||
dsn, err := dsnWithDB(lagoonDSN, name)
|
||||
|
||||
@@ -58,9 +58,6 @@ func startLighthousePostgres(ctx context.Context) error {
|
||||
postgres.WithUsername("lighthouse"),
|
||||
postgres.WithPassword("lighthouse"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
testcontainers.WithEnv(map[string]string{
|
||||
"POSTGRES_INITDB_ARGS": "--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8",
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -106,14 +103,14 @@ func adminDB(t *testing.T) *sql.DB {
|
||||
return lhSQL
|
||||
}
|
||||
|
||||
// migratedDB returns a dedicated ICU pl-PL database migrated with
|
||||
// migratedDB returns a dedicated database migrated with
|
||||
// lagoon.Migrate (River v7 and summer_jobs) plus its DSN.
|
||||
func migratedDB(t *testing.T) (*sql.DB, string) {
|
||||
t.Helper()
|
||||
admin := adminDB(t)
|
||||
ctx := t.Context()
|
||||
name := fmt.Sprintf("lighthouse_%d", dbSeq.Add(1))
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+name+` TEMPLATE template0 ENCODING 'UTF8' LOCALE_PROVIDER icu ICU_LOCALE 'pl-PL'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
if _, err := admin.ExecContext(ctx, `CREATE DATABASE `+name+` TEMPLATE template0 ENCODING 'UTF8'`); err != nil && !strings.Contains(err.Error(), "already exists") {
|
||||
t.Fatalf("create %s: %v", name, err)
|
||||
}
|
||||
dsn, err := dsnWithDB(lhDSN, name)
|
||||
|
||||
@@ -249,10 +249,8 @@ stage_postgres() {
|
||||
local dir
|
||||
dir="$(phase8_workdir)"
|
||||
PHASE8_PG_CONTAINER="phase8-pg-$$"
|
||||
# ICU pl-PL locale matches every other Go test's testcontainers Postgres
|
||||
# in this project (e.g. plugins/golem15/fonoteka/updates's own
|
||||
# startOAuthUpdatesPostgres): lagoon.Use refuses any other locale
|
||||
# provider/locale at boot.
|
||||
# The ICU pl-PL locale matches the application's recommended database;
|
||||
# lagoon itself no longer checks the database locale.
|
||||
docker run -d --rm --name "$PHASE8_PG_CONTAINER" \
|
||||
-e POSTGRES_PASSWORD=phase8 -e POSTGRES_DB=fonoteka_phase8 \
|
||||
-e POSTGRES_INITDB_ARGS="--locale-provider=icu --icu-locale=pl-PL --encoding=UTF8" \
|
||||
|
||||
Reference in New Issue
Block a user