fix(11-05): sync single-statement and plain-transaction writes, type engine status errors

- pin the sync callbacks before gorm:commit_or_rollback_transaction: an
  After-only anchor is appended past the commit and lagoon's after-commit
  flush, so single-statement writes never synced
- give the gate and the document builder a clean session: Session with NewDB
  and a Context clones the write's statement, and a later WithContext queried
  through the written model's table
- typesense.StatusError carries method, path and status, never the body
- README: sync semantics, the three gates, delete on soft delete, and the
  SQL re-gate required of SearchIDs callers
This commit is contained in:
Jakub Zych
2026-09-30 13:05:10 +02:00
parent 3e1f3e6a1b
commit 9543e6508c
5 changed files with 63 additions and 9 deletions

View File

@@ -21,7 +21,27 @@ The package itself knows no search server. An engine package registers itself fr
- The `beachcomber.Engine` driver contract: `Name`, `Configured`, `Upsert`, `Delete`, `Flush` and `SearchIDs` with a `beachcomber.Query`.
- GORM callbacks `beachcomber.CallbackAfterCreate`, `beachcomber.CallbackAfterUpdate` and `beachcomber.CallbackAfterDelete`, installed once per `*gorm.DB`, register the sync with `lagoon.AfterCommit`.
- `beachcomber.Service.Sync` and `beachcomber.Service.Remove` run the same gated path on demand, for reindex tooling, and return the error instead of logging it.
- Typesense engine (`typesense.Engine`, engine name `typesense`): the `X-TYPESENSE-API-KEY` header on every request; `Upsert` reads the collection and creates it from the schema on 404, then imports JSON lines with `action=upsert`; `Delete` and `Flush` treat 404 as success; `SearchIDs` returns `hits[].document.id`.
- Typesense engine (`typesense.Engine`, engine name `typesense`):
- Every request carries the `X-TYPESENSE-API-KEY` header.
- `Upsert` reads the collection and creates it from the schema on 404. A 409 on create counts as success, and a model without a schema gets an auto-typed collection. It then imports the documents as JSON lines (`Content-Type: text/plain`) with `action=upsert`. Typesense answers 200 even when a document fails, so every answer line is checked and any `"success":false` line is an error.
- `Delete` and `Flush` treat 404 as success.
- `SearchIDs` sends `q` (default `*`), `query_by`, `filter_by`, `sort_by`, `page` and `per_page`, and returns `hits[].document.id` in order.
- Ids and index names are path-escaped.
- A non-2xx answer is a `typesense.StatusError` with the method, path and status, never the answer body.
## Sync semantics
- **After commit.** The callbacks register the sync with `lagoon.AfterCommit`. Inside `lagoon.Transaction` it runs after that transaction commits, and not at all when it rolls back. A single-statement write, for which GORM opens its own transaction, syncs after that commit and not when the write fails. Inside a plain `gorm` transaction there is no commit hook, so the sync runs immediately through the transaction's handle. Its reads run in a savepoint, so a failed read never aborts the caller's transaction.
- **Inline and non-fatal.** The sync runs in the writing goroutine, after the commit, so a create followed by a search sees the document. Every engine request is bounded by the engine's timeout (`search.typesense.connection_timeout_seconds`), and the caller's context cancellation does not abandon it. A failure, a timeout or a panic is logged at Warn as `search: sync failed` with the index, key and operation. The write is already committed and stays so. The log never carries the document or the API key.
- **Three gates, before any request.** Nothing is sent when:
1. the engine is not configured (the `null` engine, or Typesense with an empty `search.typesense.api_key`);
2. no `*gorm.DB` is published on the app (a fresh install);
3. the application `beachcomber.Gate` reports off. A gate must treat a read error as off.
When the engine is not configured, the callbacks do not even register work.
- **Reload, then decide.** The row is reloaded by primary key, including soft-deleted rows. A delete, a row that is gone, a soft-deleted row (a set `gorm.DeletedAt`) or a row whose `ShouldBeSearchable` is false has its document deleted. Restoring a soft-deleted row is an ordinary update and indexes it again. An error from `ToSearchableArray` is logged and nothing is sent, which lets a model refuse a document that would break scoping. A document without an `id` gets the key.
- **Rows only.** A statement without a primary key value, such as `Model(&T{}).Where(…).Updates(…)` or `Delete(&T{}, id)`, cannot be synced row by row and is skipped. Bulk paths call `beachcomber.Service.Sync` or `beachcomber.Service.Remove` per row, or reindex.
- **Candidates, not answers.** `beachcomber.Engine.SearchIDs` returns candidate ids from an external index that may be stale. Callers must re-gate every id in SQL (ownership, visibility, soft deletes) before they expose a row. An empty result is an empty list, never an error.
## Usage
@@ -120,6 +140,7 @@ ids, err := svc.Engine().SearchIDs(ctx, svc.IndexName(&models.Post{}), beachcomb
|------------|-------------|
| `typesense.Config`, `typesense.LoadConfig` | The `search.typesense.*` settings with their defaults; `BaseURL` is `{protocol}://{host}:{port}{path}`. |
| `typesense.Engine`, `typesense.New` | The `beachcomber.Engine`, with `Config`. |
| `typesense.StatusError` | A non-2xx answer: `Method`, `Path`, `Code` and `StatusCode()`. |
| `typesense.DriverName` | `typesense`. |
| `typesense.DefaultHost`, `typesense.DefaultPort`, `typesense.DefaultProtocol`, `typesense.DefaultConnectionTimeout`, `typesense.DefaultImportAction` | Defaults of the configuration keys. |