feat(12-01): report the search engine's found count and field weights

- optional beachcomber.PageSearcher returns a page of candidate ids plus
  the engine's found count; beachcomber.SearchPage falls back to
  SearchIDs for engines without it, so Engine is unchanged
- Query.QueryByWeights is sent to Typesense as query_by_weights; a
  mismatched weight list or a page above typesense.MaxPerPage (250) is
  refused before any request
This commit is contained in:
Jakub Zych
2026-10-02 11:37:01 +02:00
parent e06e0cc8bf
commit 3ae49bb6f4
7 changed files with 253 additions and 17 deletions

View File

@@ -18,14 +18,15 @@ The package itself knows no search server. An engine package registers itself fr
- Engine selection by `search.driver`: `null` (the default) or a registered engine such as `typesense`. An unknown name is a boot error that lists the registered engines. Third-party engines register with `beachcomber.RegisterEngine` and a `beachcomber.EngineFactory`; a duplicate name panics at init.
- The `beachcomber.Searchable` model contract: `SearchableAs` (the index name, prefixed with `search.prefix`), `ToSearchableArray(ctx, db)` (the document, built from the committed row and free to query related rows) and `ShouldBeSearchable`. A model can also implement `beachcomber.IndexSchemaProvider` (the schema the engine creates a missing index with) and `beachcomber.SearchKeyer` (a document key other than the decimal primary key).
- The `beachcomber.Engine` driver contract: `Name`, `Configured`, `Upsert`, `Delete`, `Flush` and `SearchIDs` with a `beachcomber.Query`.
- The `beachcomber.Engine` driver contract: `Name`, `Configured`, `Upsert`, `Delete`, `Flush` and `SearchIDs` with a `beachcomber.Query`. `beachcomber.Query.QueryByWeights` ranks the `QueryBy` fields, one weight per field.
- Search totals: the optional `beachcomber.PageSearcher` interface returns a `beachcomber.SearchResult`, one page of ids plus `Found`, the number of documents the engine matched. Call it through `beachcomber.SearchPage`, which falls back to `SearchIDs` (with `Found` set to the number of ids) for an engine that does not implement it; the `null` engine returns no ids and a zero count.
- GORM callbacks `beachcomber.CallbackAfterCreate`, `beachcomber.CallbackAfterUpdate` and `beachcomber.CallbackAfterDelete`, installed once per `*gorm.DB`, register the sync with `lagoon.AfterCommit`.
- `beachcomber.Service.Sync` and `beachcomber.Service.Remove` run the same gated path on demand, for reindex tooling, and return the error instead of logging it.
- Typesense engine (`typesense.Engine`, engine name `typesense`):
- Every request carries the `X-TYPESENSE-API-KEY` header.
- `Upsert` reads the collection and creates it from the schema on 404. A 409 on create counts as success, and a model without a schema gets an auto-typed collection. It then imports the documents as JSON lines (`Content-Type: text/plain`) with `action=upsert`. Typesense answers 200 even when a document fails, so every answer line is checked and any `"success":false` line is an error.
- `Delete` and `Flush` treat 404 as success.
- `SearchIDs` sends `q` (default `*`), `query_by`, `filter_by`, `sort_by`, `page` and `per_page`, and returns `hits[].document.id` in order.
- `SearchIDs` and `SearchPage` send one request with `q` (default `*`), `query_by`, `query_by_weights`, `filter_by`, `sort_by`, `page` and `per_page`, and return `hits[].document.id` in order; `SearchPage` adds the answer's `found`. A weight list whose length differs from `QueryBy`, or a `PerPage` above `typesense.MaxPerPage` (250, Typesense's limit), is an error before any request; callers page instead.
- Ids and index names are path-escaped.
- A non-2xx answer is a `typesense.StatusError` with the method, path and status, never the answer body.
@@ -41,7 +42,7 @@ The package itself knows no search server. An engine package registers itself fr
When the engine is not configured, the callbacks do not even register work.
- **Reload, then decide.** The row is reloaded by primary key, including soft-deleted rows. A delete, a row that is gone, a soft-deleted row (a set `gorm.DeletedAt`) or a row whose `ShouldBeSearchable` is false has its document deleted. Restoring a soft-deleted row is an ordinary update and indexes it again. An error from `ToSearchableArray` is logged and nothing is sent, which lets a model refuse a document that would break scoping. A document without an `id` gets the key.
- **Rows only.** A statement without a primary key value, such as `Model(&T{}).Where(…).Updates(…)` or `Delete(&T{}, id)`, cannot be synced row by row and is skipped. Bulk paths call `beachcomber.Service.Sync` or `beachcomber.Service.Remove` per row, or reindex.
- **Candidates, not answers.** `beachcomber.Engine.SearchIDs` returns candidate ids from an external index that may be stale. Callers must re-gate every id in SQL (ownership, visibility, soft deletes) before they expose a row. An empty result is an empty list, never an error.
- **Candidates, not answers.** `beachcomber.Engine.SearchIDs` returns candidate ids from an external index that may be stale. Callers must re-gate every id in SQL (ownership, visibility, soft deletes) before they expose a row. An empty result is an empty list, never an error. The same holds for `beachcomber.SearchResult.Found`: it counts documents in the index, including stale ones, so a total shown to a user is recounted in SQL over re-gated ids, never taken from the engine.
## Usage
@@ -128,7 +129,10 @@ ids, err := svc.Engine().SearchIDs(ctx, svc.IndexName(&models.Post{}), beachcomb
| `beachcomber.IndexSchemaProvider` | `SearchIndexSchema()`: the schema a missing index is created with. |
| `beachcomber.SearchKeyer` | `SearchKey()`: replaces the decimal primary key as the document key. |
| `beachcomber.Engine` | `Name`, `Configured`, `Upsert`, `Delete`, `Flush`, `SearchIDs`. |
| `beachcomber.Query` | `Q`, `QueryBy`, `FilterBy`, `SortBy`, `Page`, `PerPage`. |
| `beachcomber.Query` | `Q`, `QueryBy`, `QueryByWeights`, `FilterBy`, `SortBy`, `Page`, `PerPage`. |
| `beachcomber.PageSearcher` | Optional engine interface: `SearchPage(ctx, index, q)` returns a page of ids and the engine's found count. |
| `beachcomber.SearchResult` | `IDs` (candidates) and `Found` (documents the engine matched). |
| `beachcomber.SearchPage(ctx, engine, index, q)` | Calls `PageSearcher` when the engine has it, else `SearchIDs` with `Found` set to the number of ids. |
| `beachcomber.Gate`, `beachcomber.GateFunc` | The application kill-switch: `Enabled(ctx, db) bool`. |
| `beachcomber.EngineFactory`, `beachcomber.RegisterEngine(name, factory)` | Registers an engine from an `init` function. |
| `beachcomber.NullEngine`, `beachcomber.DefaultDriver` | The name of the built-in engine that indexes nothing, and the default of `search.driver`. |
@@ -139,7 +143,8 @@ ids, err := svc.Engine().SearchIDs(ctx, svc.IndexName(&models.Post{}), beachcomb
| Identifier | Description |
|------------|-------------|
| `typesense.Config`, `typesense.LoadConfig` | The `search.typesense.*` settings with their defaults; `BaseURL` is `{protocol}://{host}:{port}{path}`. |
| `typesense.Engine`, `typesense.New` | The `beachcomber.Engine`, with `Config`. |
| `typesense.Engine`, `typesense.New` | The `beachcomber.Engine` and `beachcomber.PageSearcher`, with `Config`. |
| `typesense.MaxPerPage` | 250, the largest page a search may ask for. |
| `typesense.StatusError` | A non-2xx answer: `Method`, `Path`, `Code` and `StatusCode()`. |
| `typesense.DriverName` | `typesense`. |
| `typesense.DefaultHost`, `typesense.DefaultPort`, `typesense.DefaultProtocol`, `typesense.DefaultConnectionTimeout`, `typesense.DefaultImportAction` | Defaults of the configuration keys. |