feat(beachcomber): search highlights and field projection (quick-261108-pe4)

- Additive: Query.IncludeFields, ExcludeFields and Highlight; left empty, the Typesense request is the same as before. - SearchResult.Hits holds one Hit per id, in step with IDs; SearchPage always fills it. - Snippets are plain-text Segment runs split at private-use highlight tags, never HTML; MaxSnippetsPerArray is applied client-side. - Optional HighlightSearcher capability, checked with SupportsHighlights (true for Typesense, false for null and third-party engines). - README and docs: Highlights sections with the rule that a caller drops the hit of every id failing the SQL re-check before showing a snippet. Targets framework v0.6.4 (additive, no breaking change).
2026-11-08 18:31:27 +01:00
parent 6f9b77033f
commit 7819c6746c
9 changed files with 756 additions and 27 deletions
--- a/modules/beachcomber/README.md
+++ b/modules/beachcomber/README.md
@@ -30,8 +30,9 @@ Use cases:

 - Engine selection by `search.driver`: `null` (the default) or a registered engine such as `typesense`. An unknown name is a boot error that lists the registered engines. Third-party engines register with `beachcomber.RegisterEngine` and a `beachcomber.EngineFactory`; a duplicate name panics at init.
 - The `beachcomber.Searchable` model contract: `SearchableAs` (the index name, prefixed with `search.prefix`), `ToSearchableArray(ctx, db)` (the document, built from the committed row and free to query related rows) and `ShouldBeSearchable`. A model can also implement `beachcomber.IndexSchemaProvider` (the schema the engine creates a missing index with) and `beachcomber.SearchKeyer` (a document key other than the decimal primary key).
- The `beachcomber.Engine` driver contract: `Name`, `Configured`, `Upsert`, `Delete`, `Flush` and `SearchIDs` with a `beachcomber.Query`. `beachcomber.Query.QueryByWeights` ranks the `QueryBy` fields, one weight per field.
+- The `beachcomber.Engine` driver contract: `Name`, `Configured`, `Upsert`, `Delete`, `Flush` and `SearchIDs` with a `beachcomber.Query`. `beachcomber.Query.QueryByWeights` ranks the `QueryBy` fields, one weight per field. `beachcomber.Query.IncludeFields` and `beachcomber.Query.ExcludeFields` shrink the answer, and `beachcomber.Query.Highlight` asks for snippets.
 - Search totals: the optional `beachcomber.PageSearcher` interface returns a `beachcomber.SearchResult`, one page of ids plus `Found`, the number of documents the engine matched. Call it through `beachcomber.SearchPage`, which falls back to `SearchIDs` (with `Found` set to the number of ids) for an engine that does not implement it; the `null` engine returns no ids and a zero count.
+- Search highlights: `beachcomber.SearchResult.Hits` holds one `beachcomber.Hit` per id, in step with `IDs`, whose snippets are runs of plain stored text, never HTML. The optional `beachcomber.HighlightSearcher` interface marks an engine that fills them; ask through `beachcomber.SupportsHighlights`. See [Highlights](#highlights).
 - Index lifecycle for reindex tooling: the optional `beachcomber.IndexDropper` interface drops an index and reports whether it existed, so a command can tell a dropped index from one that was already absent; call it through `beachcomber.DropIndex`, which falls back to `Flush` (reporting existed) for other engines. The optional `beachcomber.IndexEnsurer` creates an empty index from its schema; `beachcomber.EnsureIndex` calls it and is a no-op for other engines. A full reindex calls it first because an `Upsert` of zero documents creates nothing.
 - GORM callbacks `beachcomber.CallbackAfterCreate`, `beachcomber.CallbackAfterUpdate` and `beachcomber.CallbackAfterDelete`, installed once per `*gorm.DB`, register the sync with `lagoon.AfterCommit`.
 - `beachcomber.Service.Sync` and `beachcomber.Service.Remove` run the same gated path on demand, for reindex tooling, and return the error instead of logging it.
@@ -41,6 +42,7 @@ Use cases:
  - `Delete` and `Flush` treat 404 as success.
  - `DropIndex` sends `DELETE /collections/{index}` and reports existed true on 2xx and false on 404; `EnsureIndex` runs the same read-then-create step as `Upsert`, so a second call sends no create.
  - `SearchIDs` and `SearchPage` send one request with `q` (default `*`), `query_by`, `query_by_weights`, `filter_by`, `sort_by`, `page` and `per_page`, and return `hits[].document.id` in order; `SearchPage` adds the answer's `found`. A weight list whose length differs from `QueryBy`, or a `PerPage` above `typesense.MaxPerPage` (250, Typesense's limit), is an error before any request; callers page instead.
+  - `include_fields` (always with `id`), `exclude_fields`, `highlight_fields`, `highlight_affix_num_tokens` and private-use highlight start and end tags are sent only when the query sets them. The `highlights` array of each hit is split into segments at those tags, with `indices` filling `ArrayIndex` for array fields. `MaxSnippetsPerArray` is applied client-side, because Typesense has no such parameter. An `ExcludeFields` that names `id`, or a negative highlight setting, is an error before any request.
  - Ids and index names are path-escaped.
  - A non-2xx answer is a `typesense.StatusError` with the method, path and status, never the answer body.

@@ -58,6 +60,15 @@ Use cases:
 - **Rows only.** A statement without a primary key value, such as `Model(&T{}).Where(…).Updates(…)` or `Delete(&T{}, id)`, cannot be synced row by row and is skipped. Bulk paths call `beachcomber.Service.Sync` or `beachcomber.Service.Remove` per row, or reindex.
 - **Candidates, not answers.** `beachcomber.Engine.SearchIDs` returns candidate ids from an external index that may be stale. Callers must re-gate every id in SQL (ownership, visibility, soft deletes) before they expose a row. An empty result is an empty list, never an error. The same holds for `beachcomber.SearchResult.Found`: it counts documents in the index, including stale ones, so a total shown to a user is recounted in SQL over re-gated ids, never taken from the engine.

+## Highlights
+
+- `beachcomber.Query.Highlight` asks for snippets: `Fields` (empty is the engine default, the `QueryBy` fields for Typesense), `AffixTokens` (tokens of context around a match, 0 is the engine default) and `MaxSnippetsPerArray` (0 is no cap). There is no option to highlight a whole field.
+- `beachcomber.SearchResult.Hits` holds one `beachcomber.Hit` per id, in `IDs` order (`Hits[i].ID == IDs[i]`), including hits without highlights. An engine called directly may leave it nil; `beachcomber.SearchPage` always fills it.
+- Each `beachcomber.Snippet` has `Field`, `ArrayIndex` (the element of an array field, -1 for a non-array field) and `beachcomber.Segment` runs with `Text` and `Match`. The text is plain stored text, verbatim, never HTML: escape `Text` when rendering and never parse markup out of it.
+- **Highlights belong to candidates.** A snippet is cut from the indexed document, so it can quote a draft, a deleted record or another user's data. Drop the `Hit` of every id that fails the SQL privacy re-check before showing any snippet; otherwise a snippet leaks the private document.
+- `beachcomber.SupportsHighlights` tells whether the engine fills highlights (true for Typesense, false for `null` and for engines without `beachcomber.HighlightSearcher`). When it is false, build snippets from the rows loaded after the re-check.
+- `beachcomber.Query.IncludeFields` and `beachcomber.Query.ExcludeFields` shrink the answer. A field left out of the answer stays searchable and highlightable, so a large text field can be stored, highlighted and still excluded from the answer. The document id is always returned.
+
 ## Usage

 An application selects the engine in `config/search.yaml`:
@@ -143,10 +154,14 @@ ids, err := svc.Engine().SearchIDs(ctx, svc.IndexName(&models.Post{}), beachcomb
 | `beachcomber.IndexSchemaProvider` | `SearchIndexSchema()`: the schema a missing index is created with. |
 | `beachcomber.SearchKeyer` | `SearchKey()`: replaces the decimal primary key as the document key. |
 | `beachcomber.Engine` | `Name`, `Configured`, `Upsert`, `Delete`, `Flush`, `SearchIDs`. |
-| `beachcomber.Query` | `Q`, `QueryBy`, `QueryByWeights`, `FilterBy`, `SortBy`, `Page`, `PerPage`. |
+| `beachcomber.Query` | `Q`, `QueryBy`, `QueryByWeights`, `FilterBy`, `SortBy`, `Page`, `PerPage`, `IncludeFields`, `ExcludeFields`, `Highlight`. |
+| `beachcomber.Highlight` | `Fields`, `AffixTokens`, `MaxSnippetsPerArray`: the snippets a query asks for. |
 | `beachcomber.PageSearcher` | Optional engine interface: `SearchPage(ctx, index, q)` returns a page of ids and the engine's found count. |
-| `beachcomber.SearchResult` | `IDs` (candidates) and `Found` (documents the engine matched). |
-| `beachcomber.SearchPage(ctx, engine, index, q)` | Calls `PageSearcher` when the engine has it, else `SearchIDs` with `Found` set to the number of ids. |
+| `beachcomber.SearchResult` | `IDs` (candidates), `Found` (documents the engine matched) and `Hits` (one per id, in the same order). |
+| `beachcomber.Hit`, `beachcomber.Snippet`, `beachcomber.Segment` | A hit's `ID` and `Highlights`; a snippet's `Field`, `ArrayIndex` and `Segments`; a segment's plain `Text` and `Match`. |
+| `beachcomber.SearchPage(ctx, engine, index, q)` | Calls `PageSearcher` when the engine has it, else `SearchIDs` with `Found` set to the number of ids; always fills `Hits` in step with `IDs`. |
+| `beachcomber.HighlightSearcher` | Optional engine interface: a `PageSearcher` with `SupportsHighlights()`, filling `Hit.Highlights`. |
+| `beachcomber.SupportsHighlights(e)` | Whether the engine implements `HighlightSearcher` and answers true. |
 | `beachcomber.IndexDropper` | Optional engine interface: `DropIndex(ctx, index)` drops the index and reports whether it existed. |
 | `beachcomber.DropIndex(ctx, engine, index)` | Calls `IndexDropper` when the engine has it, else `Flush` and reports existed true. |
 | `beachcomber.IndexEnsurer` | Optional engine interface: `EnsureIndex(ctx, index, schema)` creates a missing index, empty, from its schema. |
@@ -161,7 +176,7 @@ ids, err := svc.Engine().SearchIDs(ctx, svc.IndexName(&models.Post{}), beachcomb
 | Identifier | Description |
 |------------|-------------|
 | `typesense.Config`, `typesense.LoadConfig` | The `search.typesense.*` settings with their defaults; `BaseURL` is `{protocol}://{host}:{port}{path}`. |
-| `typesense.Engine`, `typesense.New` | The `beachcomber.Engine`, `beachcomber.PageSearcher`, `beachcomber.IndexDropper` and `beachcomber.IndexEnsurer`, with `Config`. |
+| `typesense.Engine`, `typesense.New` | The `beachcomber.Engine`, `beachcomber.PageSearcher`, `beachcomber.HighlightSearcher`, `beachcomber.IndexDropper` and `beachcomber.IndexEnsurer`, with `Config`. |
 | `typesense.MaxPerPage` | 250, the largest page a search may ask for. |
 | `typesense.StatusError` | A non-2xx answer: `Method`, `Path`, `Code` and `StatusCode()`. |
 | `typesense.DriverName` | `typesense`. |
--- a/modules/beachcomber/engines.go
+++ b/modules/beachcomber/engines.go
@@ -62,7 +62,8 @@ func init() {
 }

 // nullEngine is never configured, so sync is skipped before it is called.
-// Every method is a no-op; SearchIDs and SearchPage return no ids.
+// Every method is a no-op; SearchIDs and SearchPage return no ids. It does
+// not implement HighlightSearcher.
 type nullEngine struct{}

 func (nullEngine) Name() string     { return NullEngine }
@@ -76,5 +77,5 @@ func (nullEngine) SearchIDs(context.Context, string, Query) ([]string, error) {
 	return []string{}, nil
 }
 func (nullEngine) SearchPage(context.Context, string, Query) (SearchResult, error) {
-	return SearchResult{IDs: []string{}}, nil
+	return SearchResult{IDs: []string{}, Hits: []Hit{}}, nil
 }
--- a/modules/beachcomber/searchable.go
+++ b/modules/beachcomber/searchable.go
@@ -75,6 +75,32 @@ type Query struct {
 	Page int
 	// PerPage is the page size; 0 leaves the engine default.
 	PerPage int
+	// IncludeFields and ExcludeFields only limit which document fields the
+	// engine sends back: a field left out of the answer stays searchable
+	// and highlightable. Empty keeps the engine default (every stored
+	// field). The document key is always returned, because the ids are the
+	// result. Engines that cannot project ignore them.
+	IncludeFields []string
+	ExcludeFields []string
+	// Highlight asks for snippets of the matches in SearchResult.Hits. Nil
+	// asks for no highlights and keeps the request as it was. Engines
+	// without HighlightSearcher ignore it.
+	Highlight *Highlight
+}
+
+// Highlight configures the snippets an engine returns for a Query.
+// Snippets are always cut around the matches; there is deliberately no
+// option to highlight a whole field.
+type Highlight struct {
+	// Fields are the fields to highlight. Empty is the engine default,
+	// which is the QueryBy fields for Typesense.
+	Fields []string
+	// AffixTokens is the number of tokens of context on each side of a
+	// match; 0 is the engine default.
+	AffixTokens int
+	// MaxSnippetsPerArray is the most snippets kept for one array field;
+	// 0 means no cap.
+	MaxSnippetsPerArray int
 }

 // SearchResult is one page of candidate ids and the number of documents
@@ -83,6 +109,36 @@ type Query struct {
 type SearchResult struct {
 	IDs   []string
 	Found int
+	// Hits has one entry per id, in the same order as IDs
+	// (Hits[i].ID == IDs[i]). An engine called directly may leave it nil;
+	// SearchPage always fills it. Like the ids, hits are candidates: drop
+	// the Hit of every id the SQL re-check rejects before showing a
+	// snippet.
+	Hits []Hit
+}
+
+// Hit is the engine's answer for one id beyond the id itself.
+type Hit struct {
+	ID string
+	// Highlights is nil when the hit has none.
+	Highlights []Snippet
+}
+
+// Snippet is one highlighted excerpt of a field.
+type Snippet struct {
+	Field string
+	// ArrayIndex is the element index for an array field, and -1 for a
+	// non-array field or when the engine did not say which element.
+	ArrayIndex int
+	Segments   []Segment
+}
+
+// Segment is a run of a snippet that either matched the query or not.
+// Text is plain stored text, verbatim, never HTML or markup. It can
+// contain < or &, so the caller escapes it when rendering.
+type Segment struct {
+	Text  string
+	Match bool
 }

 // PageSearcher is implemented by an engine that can report how many
@@ -92,18 +148,51 @@ type PageSearcher interface {
 	SearchPage(ctx context.Context, index string, q Query) (SearchResult, error)
 }

+// HighlightSearcher is implemented by an engine that can return snippets
+// of the matches. It is optional so that existing engines stay valid. An
+// engine that implements it and answers true fills Hit.Highlights for a
+// Query with Highlight; ask through SupportsHighlights.
+type HighlightSearcher interface {
+	PageSearcher
+	SupportsHighlights() bool
+}
+
+// SupportsHighlights reports whether e fills Hit.Highlights: true only when
+// e implements HighlightSearcher and its SupportsHighlights answers true. A
+// nil e is false. When it is false, an app builds its own snippets from the
+// rows it loads.
+func SupportsHighlights(e Engine) bool {
+	hs, ok := e.(HighlightSearcher)
+	return ok && hs.SupportsHighlights()
+}
+
 // SearchPage asks e for one page of q: through PageSearcher when e
 // implements it, otherwise through SearchIDs with Found set to the number
-// of ids returned.
+// of ids returned. On success Hits is always in step with IDs: when the
+// engine left it nil, it is one Hit without highlights per id (an empty,
+// non-nil slice for zero ids); hits the engine filled are kept as they are.
 func SearchPage(ctx context.Context, e Engine, index string, q Query) (SearchResult, error) {
+	var res SearchResult
 	if ps, ok := e.(PageSearcher); ok {
-		return ps.SearchPage(ctx, index, q)
+		r, err := ps.SearchPage(ctx, index, q)
+		if err != nil {
+			return r, err
+		}
+		res = r
+	} else {
+		ids, err := e.SearchIDs(ctx, index, q)
+		if err != nil {
+			return SearchResult{}, err
+		}
+		res = SearchResult{IDs: ids, Found: len(ids)}
 	}
-	ids, err := e.SearchIDs(ctx, index, q)
-	if err != nil {
-		return SearchResult{}, err
+	if res.Hits == nil {
+		res.Hits = make([]Hit, len(res.IDs))
+		for i, id := range res.IDs {
+			res.Hits[i] = Hit{ID: id}
+		}
 	}
-	return SearchResult{IDs: ids, Found: len(ids)}, nil
+	return res, nil
 }

 // IndexDropper is implemented by an engine that can tell whether the index
--- a/modules/beachcomber/searchpage_test.go
+++ b/modules/beachcomber/searchpage_test.go
@@ -45,7 +45,7 @@ func TestSearchPageFallsBackToSearchIDs(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	if !reflect.DeepEqual(res, SearchResult{IDs: []string{"3", "1"}, Found: 2}) {
+	if !reflect.DeepEqual(res, SearchResult{IDs: []string{"3", "1"}, Found: 2, Hits: []Hit{{ID: "3"}, {ID: "1"}}}) {
 		t.Fatalf("res = %+v", res)
 	}
 	if !reflect.DeepEqual(e.got, q) {
@@ -74,11 +74,63 @@ func TestSearchPageNullEngine(t *testing.T) {
 		t.Fatal("the null engine implements PageSearcher")
 	}
 	res, err := SearchPage(context.Background(), e, "posts", Query{})
-	if err != nil || res.Found != 0 || res.IDs == nil || len(res.IDs) != 0 {
+	if err != nil || res.Found != 0 || res.IDs == nil || len(res.IDs) != 0 || res.Hits == nil || len(res.Hits) != 0 {
 		t.Fatalf("res = %+v err = %v", res, err)
 	}
 }

+// highlightEngine implements HighlightSearcher with a configurable answer
+// and can return its own hits.
+type highlightEngine struct {
+	pagedEngine
+	supports bool
+	hits     []Hit
+}
+
+func (e *highlightEngine) SupportsHighlights() bool { return e.supports }
+
+func (e *highlightEngine) SearchPage(_ context.Context, _ string, q Query) (SearchResult, error) {
+	e.got = q
+	return SearchResult{IDs: e.ids, Found: e.found, Hits: e.hits}, nil
+}
+
+func TestSupportsHighlights(t *testing.T) {
+	for name, e := range map[string]Engine{
+		"null":     nullEngine{},
+		"ids-only": &idsOnlyEngine{},
+		"paged":    &pagedEngine{},
+		"nil":      nil,
+	} {
+		if SupportsHighlights(e) {
+			t.Fatalf("%s: SupportsHighlights = true", name)
+		}
+	}
+	if !SupportsHighlights(&highlightEngine{supports: true}) {
+		t.Fatal("an engine answering true supports highlights")
+	}
+	if SupportsHighlights(&highlightEngine{supports: false}) {
+		t.Fatal("an engine answering false does not support highlights")
+	}
+}
+
+func TestSearchPageFillsHits(t *testing.T) {
+	ctx := context.Background()
+	res, err := SearchPage(ctx, &pagedEngine{idsOnlyEngine: idsOnlyEngine{ids: []string{"7"}}, found: 1}, "posts", Query{})
+	if err != nil || !reflect.DeepEqual(res.Hits, []Hit{{ID: "7"}}) {
+		t.Fatalf("hits = %+v err = %v", res.Hits, err)
+	}
+	own := []Hit{{ID: "7", Highlights: []Snippet{{Field: "title", ArrayIndex: -1, Segments: []Segment{{Text: "x", Match: true}}}}}}
+	he := &highlightEngine{pagedEngine: pagedEngine{idsOnlyEngine: idsOnlyEngine{ids: []string{"7"}}}, supports: true, hits: own}
+	res, err = SearchPage(ctx, he, "posts", Query{Highlight: &Highlight{}})
+	if err != nil || !reflect.DeepEqual(res.Hits, own) {
+		t.Fatalf("own hits = %+v err = %v", res.Hits, err)
+	}
+	res, err = SearchPage(ctx, &pagedEngine{}, "posts", Query{})
+	if err != nil || res.Hits == nil || len(res.Hits) != 0 {
+		t.Fatalf("zero ids: hits = %#v err = %v", res.Hits, err)
+	}
+}
+
 // flushEngine records Flush calls and can fail them.
 type flushEngine struct {
 	idsOnlyEngine
--- a/modules/beachcomber/typesense/engine.go
+++ b/modules/beachcomber/typesense/engine.go
@@ -10,6 +10,7 @@ import (
 	"maps"
 	"net/http"
 	"net/url"
+	"slices"
 	"strconv"
 	"strings"

@@ -48,8 +49,9 @@ type Engine struct {

 var _ beachcomber.Engine = (*Engine)(nil)
 var (
-	_ beachcomber.IndexDropper = (*Engine)(nil)
-	_ beachcomber.IndexEnsurer = (*Engine)(nil)
+	_ beachcomber.IndexDropper      = (*Engine)(nil)
+	_ beachcomber.IndexEnsurer      = (*Engine)(nil)
+	_ beachcomber.HighlightSearcher = (*Engine)(nil)
 )

 // New returns an engine for cfg. Every request is bounded by
@@ -252,6 +254,9 @@ type searchAnswer struct {
 		Document struct {
 			ID json.RawMessage `json:"id"`
 		} `json:"document"`
+		// Highlights stays raw so an odd highlights value never makes
+		// the answer unreadable.
+		Highlights json.RawMessage `json:"highlights"`
 	} `json:"hits"`
 }

@@ -261,9 +266,11 @@ const MaxPerPage = 250

 // SearchIDs sends GET /collections/{index}/documents/search with q
 // (default "*"), query_by, query_by_weights, filter_by, sort_by, page and
-// per_page, and returns hits[].document.id in order. The ids are
-// candidates: the caller must re-check each one in SQL before exposing it.
-// No hits is an empty list.
+// per_page, and returns hits[].document.id in order. include_fields
+// (always with id), exclude_fields and, for Query.Highlight,
+// highlight_fields, highlight_affix_num_tokens and the highlight tags are
+// sent only when set. The ids are candidates: the caller must re-check
+// each one in SQL before exposing it. No hits is an empty list.
 func (e *Engine) SearchIDs(ctx context.Context, index string, q beachcomber.Query) ([]string, error) {
 	res, err := e.search(ctx, index, q)
 	if err != nil {
@@ -272,12 +279,19 @@ func (e *Engine) SearchIDs(ctx context.Context, index string, q beachcomber.Quer
 	return res.IDs, nil
 }

-// SearchPage is SearchIDs plus the found count of the answer: how many
-// documents matched in the index, not how many the caller may see.
+// SearchPage is SearchIDs plus the found count of the answer (how many
+// documents matched in the index, not how many the caller may see) and
+// Hits, one per id in the same order. With Query.Highlight set, each Hit
+// carries the snippets of the answer's highlights array, split into plain
+// text segments; without it, Highlights stays nil.
 func (e *Engine) SearchPage(ctx context.Context, index string, q beachcomber.Query) (beachcomber.SearchResult, error) {
 	return e.search(ctx, index, q)
 }

+// SupportsHighlights returns true: SearchPage fills Hits[].Highlights when
+// Query.Highlight is set. It implements beachcomber.HighlightSearcher.
+func (e *Engine) SupportsHighlights() bool { return true }
+
 func (e *Engine) search(ctx context.Context, index string, q beachcomber.Query) (beachcomber.SearchResult, error) {
 	if len(q.QueryByWeights) > 0 && len(q.QueryByWeights) != len(q.QueryBy) {
 		return beachcomber.SearchResult{}, fmt.Errorf("typesense: search %s: %d query_by_weights for %d query_by fields", index, len(q.QueryByWeights), len(q.QueryBy))
@@ -285,6 +299,19 @@ func (e *Engine) search(ctx context.Context, index string, q beachcomber.Query)
 	if q.PerPage > MaxPerPage {
 		return beachcomber.SearchResult{}, fmt.Errorf("typesense: search %s: per_page %d exceeds %d", index, q.PerPage, MaxPerPage)
 	}
+	for _, f := range q.ExcludeFields {
+		if f == "id" {
+			return beachcomber.SearchResult{}, fmt.Errorf("typesense: search %s: exclude_fields cannot leave out id, because the ids are the result", index)
+		}
+	}
+	if h := q.Highlight; h != nil {
+		if h.AffixTokens < 0 {
+			return beachcomber.SearchResult{}, fmt.Errorf("typesense: search %s: Highlight.AffixTokens %d is negative", index, h.AffixTokens)
+		}
+		if h.MaxSnippetsPerArray < 0 {
+			return beachcomber.SearchResult{}, fmt.Errorf("typesense: search %s: Highlight.MaxSnippetsPerArray %d is negative", index, h.MaxSnippetsPerArray)
+		}
+	}
 	params := url.Values{}
 	text := q.Q
 	if text == "" {
@@ -313,6 +340,27 @@ func (e *Engine) search(ctx context.Context, index string, q beachcomber.Query)
 	if q.PerPage > 0 {
 		params.Set("per_page", strconv.Itoa(q.PerPage))
 	}
+	if len(q.IncludeFields) > 0 {
+		// The Typesense docs do not promise id under include_fields.
+		fields := q.IncludeFields
+		if !slices.Contains(fields, "id") {
+			fields = append(slices.Clip(fields), "id")
+		}
+		params.Set("include_fields", strings.Join(fields, ","))
+	}
+	if len(q.ExcludeFields) > 0 {
+		params.Set("exclude_fields", strings.Join(q.ExcludeFields, ","))
+	}
+	if h := q.Highlight; h != nil {
+		params.Set("highlight_start_tag", highlightStartTag)
+		params.Set("highlight_end_tag", highlightEndTag)
+		if len(h.Fields) > 0 {
+			params.Set("highlight_fields", strings.Join(h.Fields, ","))
+		}
+		if h.AffixTokens > 0 {
+			params.Set("highlight_affix_num_tokens", strconv.Itoa(h.AffixTokens))
+		}
+	}
 	path := "/collections/" + url.PathEscape(index) + "/documents/search"
 	code, answer, err := e.do(ctx, http.MethodGet, path, params, nil, "")
 	if err != nil {
@@ -326,19 +374,28 @@ func (e *Engine) search(ctx context.Context, index string, q beachcomber.Query)
 		return beachcomber.SearchResult{}, fmt.Errorf("typesense: search %s: unreadable answer", index)
 	}
 	ids := make([]string, 0, len(res.Hits))
+	hits := make([]beachcomber.Hit, 0, len(res.Hits))
 	for _, h := range res.Hits {
 		raw := bytes.TrimSpace(h.Document.ID)
 		if len(raw) == 0 || bytes.Equal(raw, []byte("null")) {
 			continue
 		}
+		id := string(raw)
 		var s string
 		if err := json.Unmarshal(raw, &s); err == nil {
-			ids = append(ids, s)
-			continue
+			id = s
 		}
-		ids = append(ids, string(raw))
+		hit := beachcomber.Hit{ID: id}
+		// Only with Highlight set are the tags the private-use characters;
+		// otherwise the snippets carry Typesense's default mark tags and
+		// must not be read as text.
+		if q.Highlight != nil {
+			hit.Highlights = parseHighlights(h.Highlights, q.Highlight)
+		}
+		ids = append(ids, id)
+		hits = append(hits, hit)
 	}
-	return beachcomber.SearchResult{IDs: ids, Found: res.Found}, nil
+	return beachcomber.SearchResult{IDs: ids, Found: res.Found, Hits: hits}, nil
 }

 // do sends one request with the API key header and returns the status and
--- a/modules/beachcomber/typesense/example_test.go
+++ b/modules/beachcomber/typesense/example_test.go
@@ -3,10 +3,12 @@ package typesense_test
 import (
 	"context"
 	"fmt"
+	"html"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
 	"strconv"
+	"strings"
 	"time"

 	"git.golem15.com/golem15/summercms/modules/beachcomber"
@@ -47,3 +49,57 @@ func ExampleEngine_SearchIDs() {
 	// [3 1] <nil>
 	// false
 }
+
+func ExampleEngine_SearchPage() {
+	// A stand-in Typesense node. The documents carry only id and title:
+	// text is excluded from the answer, yet still highlighted.
+	node := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		q := r.URL.Query()
+		fmt.Println("exclude_fields", q.Get("exclude_fields"), "highlight_fields", q.Get("highlight_fields"),
+			"highlight_affix_num_tokens", q.Get("highlight_affix_num_tokens"))
+		fmt.Println("private-use start tag:", q.Get("highlight_start_tag") == "\ue000")
+		fmt.Fprint(w, `{"found":2,"hits":[`+
+			`{"document":{"id":"3","title":"Kind of Blue"},"highlights":[{"field":"text","snippet":"a <b> \ue000blue\ue001 note"}]},`+
+			`{"document":{"id":"8","title":"Draft"},"highlights":[{"field":"text","snippet":"private \ue000blue\ue001 draft"}]}]}`)
+	}))
+	defer node.Close()
+	u, _ := url.Parse(node.URL)
+	port, _ := strconv.Atoi(u.Port())
+	engine := typesense.New(typesense.Config{APIKey: "test-only-key", Host: u.Hostname(), Port: port, Protocol: "http"})
+
+	fmt.Println("highlights:", beachcomber.SupportsHighlights(engine))
+	res, err := beachcomber.SearchPage(context.Background(), engine, "acme_blog_posts", beachcomber.Query{
+		Q:             "blue",
+		QueryBy:       []string{"title", "text"},
+		ExcludeFields: []string{"text"},
+		Highlight:     &beachcomber.Highlight{Fields: []string{"text"}, AffixTokens: 3},
+	})
+	if err != nil {
+		fmt.Println(err)
+		return
+	}
+	// Stands in for the SQL re-check: id 8 is not visible to the caller, so
+	// its hit is dropped before any snippet is shown.
+	allowed := map[string]bool{"3": true}
+	for _, hit := range res.Hits {
+		if !allowed[hit.ID] {
+			continue
+		}
+		for _, sn := range hit.Highlights {
+			var b strings.Builder
+			for _, seg := range sn.Segments {
+				text := html.EscapeString(seg.Text) // segments are plain text
+				if seg.Match {
+					text = "<mark>" + text + "</mark>"
+				}
+				b.WriteString(text)
+			}
+			fmt.Println(hit.ID, sn.Field, sn.ArrayIndex, b.String())
+		}
+	}
+	// Output:
+	// highlights: true
+	// exclude_fields text highlight_fields text highlight_affix_num_tokens 3
+	// private-use start tag: true
+	// 3 text -1 a &lt;b&gt; <mark>blue</mark> note
+}
--- a/modules/beachcomber/typesense/highlights.go
+++ b/modules/beachcomber/typesense/highlights.go
@@ -0,0 +1,116 @@
+package typesense
+
+import (
+	"encoding/json"
+	"strings"
+
+	"git.golem15.com/golem15/summercms/modules/beachcomber"
+)
+
+// highlightStartTag and highlightEndTag are two Unicode private-use
+// characters that do not occur in normal text. The engine sends them as
+// highlight_start_tag and highlight_end_tag, so a snippet can be split into
+// segments without parsing markup.
+const (
+	highlightStartTag = ""
+	highlightEndTag   = ""
+)
+
+// splitSegments splits a snippet tagged with highlightStartTag and
+// highlightEndTag into matched and unmatched runs. Empty runs are left out
+// and the text is never trimmed, escaped or unescaped.
+func splitSegments(s string) []beachcomber.Segment {
+	var out []beachcomber.Segment
+	match := false
+	for s != "" {
+		tag := highlightStartTag
+		if match {
+			tag = highlightEndTag
+		}
+		i := strings.Index(s, tag)
+		if i < 0 {
+			out = appendSegment(out, s, match)
+			break
+		}
+		out = appendSegment(out, s[:i], match)
+		s = s[i+len(tag):]
+		match = !match
+	}
+	return out
+}
+
+func appendSegment(out []beachcomber.Segment, text string, match bool) []beachcomber.Segment {
+	if text == "" {
+		return out
+	}
+	return append(out, beachcomber.Segment{Text: text, Match: match})
+}
+
+// tsHighlight is one entry of a hit's highlights array.
+// A string field carries snippet; a string[] field carries snippets with
+// the element index of each in indices.
+type tsHighlight struct {
+	Field    string   `json:"field"`
+	Snippet  string   `json:"snippet"`
+	Snippets []string `json:"snippets"`
+	Indices  []int    `json:"indices"`
+}
+
+// parseHighlights turns a hit's highlights array into snippets, in the
+// order of the array. An entry that does not decode, or has no snippet,
+// contributes nothing; a value that is not an array gives nil.
+func parseHighlights(raw json.RawMessage, h *beachcomber.Highlight) []beachcomber.Snippet {
+	if len(raw) == 0 {
+		return nil
+	}
+	var entries []json.RawMessage
+	if err := json.Unmarshal(raw, &entries); err != nil {
+		return nil
+	}
+	var out []beachcomber.Snippet
+	for _, entry := range entries {
+		var th tsHighlight
+		if err := json.Unmarshal(entry, &th); err != nil {
+			continue
+		}
+		if len(th.Snippets) > 0 {
+			out = appendArraySnippets(out, th, h)
+			continue
+		}
+		if th.Snippet == "" {
+			continue
+		}
+		segs := splitSegments(th.Snippet)
+		if len(segs) == 0 {
+			continue
+		}
+		out = append(out, beachcomber.Snippet{Field: th.Field, ArrayIndex: -1, Segments: segs})
+	}
+	return out
+}
+
+// appendArraySnippets appends one snippet per element of an array field's
+// snippets, with ArrayIndex taken from indices (-1 when indices does not
+// cover the element). Typesense 28.0 has no parameter that caps snippets
+// per array (checked 2026-11-08), so MaxSnippetsPerArray is applied here
+// and keeps the first snippets in Typesense's order. Elements with no
+// segments are left out and do not count toward the cap.
+func appendArraySnippets(out []beachcomber.Snippet, th tsHighlight, h *beachcomber.Highlight) []beachcomber.Snippet {
+	kept := 0
+	for i, sn := range th.Snippets {
+		if h != nil && h.MaxSnippetsPerArray > 0 && kept >= h.MaxSnippetsPerArray {
+			break
+		}
+		segs := splitSegments(sn)
+		if len(segs) == 0 {
+			continue
+		}
+		idx := -1
+		if i < len(th.Indices) {
+			idx = th.Indices[i]
+		}
+		out = append(out, beachcomber.Snippet{Field: th.Field, ArrayIndex: idx, Segments: segs})
+		kept++
+	}
+	return out
+}
--- a/modules/beachcomber/typesense/highlights_test.go
+++ b/modules/beachcomber/typesense/highlights_test.go
@@ -0,0 +1,276 @@
+package typesense
+
+import (
+	"net/url"
+	"reflect"
+	"sort"
+	"strings"
+	"testing"
+
+	"git.golem15.com/golem15/summercms/modules/beachcomber"
+)
+
+const searchKey = "GET /ts/collections/posts/documents/search"
+
+// onlyQuery returns the parsed query of the single call f received.
+func onlyQuery(t *testing.T, f *fakeTypesense) url.Values {
+	t.Helper()
+	calls := f.take()
+	if len(calls) != 1 {
+		t.Fatalf("calls = %d, want 1", len(calls))
+	}
+	params, err := url.ParseQuery(calls[0].rawQuery)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return params
+}
+
+func TestTypesenseSearchSendsNoNewParamsByDefault(t *testing.T) {
+	f, e := newFake(t)
+	f.answer(searchKey, 200, `{"found":2,"hits":[`+
+		`{"document":{"id":"42"},"highlights":[{"field":"name","snippet":"a <mark>blue</mark> train","matched_tokens":["blue"]}]},`+
+		`{"document":{"id":7}}]}`)
+	q := beachcomber.Query{
+		Q:              "blue train",
+		QueryBy:        []string{"name", "artist_display", "notes"},
+		QueryByWeights: []int{10, 10, 1},
+		FilterBy:       "collection_id:=5",
+		SortBy:         "created_at:desc",
+		Page:           2,
+		PerPage:        250,
+	}
+	res, err := e.SearchPage(t.Context(), "posts", q)
+	if err != nil {
+		t.Fatal(err)
+	}
+	params := onlyQuery(t, f)
+	keys := make([]string, 0, len(params))
+	for k := range params {
+		keys = append(keys, k)
+	}
+	sort.Strings(keys)
+	want := []string{"filter_by", "page", "per_page", "q", "query_by", "query_by_weights", "sort_by"}
+	if !reflect.DeepEqual(keys, want) {
+		t.Fatalf("keys = %v, want %v", keys, want)
+	}
+	if res.Found != 2 || !reflect.DeepEqual(res.IDs, []string{"42", "7"}) {
+		t.Fatalf("res = %+v", res)
+	}
+	wantHits := []beachcomber.Hit{{ID: "42"}, {ID: "7"}}
+	if !reflect.DeepEqual(res.Hits, wantHits) {
+		t.Fatalf("hits = %+v, want %+v (mark snippets must not be read)", res.Hits, wantHits)
+	}
+}
+
+func TestTypesenseHighlightsStringSnippet(t *testing.T) {
+	f, e := newFake(t)
+	f.answer(searchKey, 200, `{"found":1,"hits":[{"document":{"id":"5"},"highlights":[{"field":"title","snippet":"a blue train","matched_tokens":["blue"]}]}]}`)
+	res, err := e.SearchPage(t.Context(), "posts", beachcomber.Query{
+		Q:         "blue",
+		QueryBy:   []string{"title"},
+		Highlight: &beachcomber.Highlight{Fields: []string{"title"}, AffixTokens: 6},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	params := onlyQuery(t, f)
+	if params.Get("highlight_fields") != "title" || params.Get("highlight_affix_num_tokens") != "6" ||
+		params.Get("highlight_start_tag") != "" || params.Get("highlight_end_tag") != "" {
+		t.Fatalf("params = %v", params)
+	}
+	if params.Has("highlight_full_fields") {
+		t.Fatal("highlight_full_fields must never be sent")
+	}
+	want := []beachcomber.Hit{{ID: "5", Highlights: []beachcomber.Snippet{{
+		Field:      "title",
+		ArrayIndex: -1,
+		Segments:   []beachcomber.Segment{{Text: "a "}, {Text: "blue", Match: true}, {Text: " train"}},
+	}}}}
+	if !reflect.DeepEqual(res.Hits, want) {
+		t.Fatalf("hits = %+v, want %+v", res.Hits, want)
+	}
+
+	// A zero Highlight sends the two tags and nothing else new.
+	if _, err := e.SearchPage(t.Context(), "posts", beachcomber.Query{Q: "blue", QueryBy: []string{"title"}, Highlight: &beachcomber.Highlight{}}); err != nil {
+		t.Fatal(err)
+	}
+	params = onlyQuery(t, f)
+	for k := range params {
+		switch k {
+		case "q", "query_by", "highlight_start_tag", "highlight_end_tag":
+		default:
+			t.Fatalf("zero Highlight sent %s (query %v)", k, params)
+		}
+	}
+	if !params.Has("highlight_start_tag") || !params.Has("highlight_end_tag") {
+		t.Fatalf("zero Highlight must send the tags: %v", params)
+	}
+}
+
+func TestTypesenseHighlightsKeepStoredText(t *testing.T) {
+	for name, snippet := range map[string]string{
+		"literal": `<b>bold</b> &amp; blue &lt;x`,
+		"escaped": `<b>bold</b> &amp; blue &lt;x`,
+	} {
+		t.Run(name, func(t *testing.T) {
+			f, e := newFake(t)
+			f.answer(searchKey, 200, `{"found":1,"hits":[{"document":{"id":"1"},"highlights":[{"field":"body","snippet":"`+snippet+`"}]}]}`)
+			res, err := e.SearchPage(t.Context(), "posts", beachcomber.Query{Q: "blue", QueryBy: []string{"body"}, Highlight: &beachcomber.Highlight{}})
+			if err != nil {
+				t.Fatal(err)
+			}
+			if len(res.Hits) != 1 || len(res.Hits[0].Highlights) != 1 {
+				t.Fatalf("hits = %+v", res.Hits)
+			}
+			want := []beachcomber.Segment{{Text: "<b>bold</b> &amp; "}, {Text: "blue", Match: true}, {Text: " &lt;x"}}
+			got := res.Hits[0].Highlights[0].Segments
+			if !reflect.DeepEqual(got, want) {
+				t.Fatalf("segments = %+v, want %+v", got, want)
+			}
+			for _, s := range got {
+				if strings.Contains(s.Text, "mark") {
+					t.Fatalf("segment carries markup: %q", s.Text)
+				}
+			}
+		})
+	}
+}
+
+func TestTypesenseHitsStayInStepWithIDs(t *testing.T) {
+	f, e := newFake(t)
+	f.answer(searchKey, 200, `{"found":5,"hits":[`+
+		`{"document":{"id":"12"},"highlights":[{"field":"title","snippet":"x"},{"field":"notes","snippet":5}]},`+
+		`{"document":{"id":null},"highlights":[{"field":"title","snippet":"y"}]},`+
+		`{"document":{},"highlights":[{"field":"title","snippet":"z"}]},`+
+		`{"document":{"id":7}},`+
+		`{"document":{"id":"3"},"highlights":[]}]}`)
+	res, err := e.SearchPage(t.Context(), "posts", beachcomber.Query{Q: "x", QueryBy: []string{"title"}, Highlight: &beachcomber.Highlight{}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !reflect.DeepEqual(res.IDs, []string{"12", "7", "3"}) || len(res.Hits) != 3 {
+		t.Fatalf("res = %+v", res)
+	}
+	for i, h := range res.Hits {
+		if h.ID != res.IDs[i] {
+			t.Fatalf("hits[%d].ID = %q, ids[%d] = %q", i, h.ID, i, res.IDs[i])
+		}
+	}
+	if len(res.Hits[0].Highlights) != 1 || res.Hits[0].Highlights[0].Field != "title" {
+		t.Fatalf("hits[0] = %+v (the undecodable notes entry is dropped alone)", res.Hits[0])
+	}
+	if len(res.Hits[1].Highlights) != 0 || len(res.Hits[2].Highlights) != 0 {
+		t.Fatalf("hits = %+v", res.Hits)
+	}
+
+	f.answer(searchKey, 200, `{"found":0,"hits":[]}`)
+	res, err = e.SearchPage(t.Context(), "posts", beachcomber.Query{Q: "x", Highlight: &beachcomber.Highlight{}})
+	if err != nil || res.Hits == nil || len(res.Hits) != 0 {
+		t.Fatalf("empty answer = %+v %v", res, err)
+	}
+}
+
+func TestTypesenseHighlightsStringArray(t *testing.T) {
+	f, e := newFake(t)
+	f.answer(searchKey, 200, `{"found":2,"hits":[`+
+		`{"document":{"id":"1"},"highlights":[{"field":"tracks","snippets":["blue note","x blue"],"indices":[2,0],"matched_tokens":[["blue"],["blue"]]}]},`+
+		`{"document":{"id":"2"},"highlights":[`+
+		`{"field":"title","snippet":"blue train"},`+
+		`{"field":"tags","snippets":["blue"]}]}]}`)
+	q := beachcomber.Query{Q: "blue", QueryBy: []string{"title", "tracks", "tags"}, Highlight: &beachcomber.Highlight{}}
+	res, err := e.SearchPage(t.Context(), "posts", q)
+	if err != nil {
+		t.Fatal(err)
+	}
+	params := onlyQuery(t, f)
+	for k := range params {
+		if strings.Contains(k, "snippet") || strings.Contains(k, "max") {
+			t.Fatalf("a snippet-cap parameter was sent: %s", k)
+		}
+	}
+	want := []beachcomber.Snippet{
+		{Field: "tracks", ArrayIndex: 2, Segments: []beachcomber.Segment{{Text: "blue", Match: true}, {Text: " note"}}},
+		{Field: "tracks", ArrayIndex: 0, Segments: []beachcomber.Segment{{Text: "x "}, {Text: "blue", Match: true}}},
+	}
+	if !reflect.DeepEqual(res.Hits[0].Highlights, want) {
+		t.Fatalf("tracks = %+v, want %+v", res.Hits[0].Highlights, want)
+	}
+	second := res.Hits[1].Highlights
+	if len(second) != 2 || second[0].Field != "title" || second[0].ArrayIndex != -1 ||
+		second[1].Field != "tags" || second[1].ArrayIndex != -1 {
+		t.Fatalf("mixed hit = %+v (order of the highlights array, -1 without indices)", second)
+	}
+
+	q.Highlight.MaxSnippetsPerArray = 1
+	res, err = e.SearchPage(t.Context(), "posts", q)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !reflect.DeepEqual(res.Hits[0].Highlights, want[:1]) {
+		t.Fatalf("capped = %+v, want %+v", res.Hits[0].Highlights, want[:1])
+	}
+	params = onlyQuery(t, f)
+	for k := range params {
+		if strings.Contains(k, "snippet") || strings.Contains(k, "max") {
+			t.Fatalf("the cap was sent as %s; it is client-side", k)
+		}
+	}
+}
+
+func TestTypesenseFieldProjection(t *testing.T) {
+	f, e := newFake(t)
+	f.answer(searchKey, 200, `{"found":1,"hits":[{"document":{"id":"9"},"highlights":[{"field":"body","snippet":"the blue one"}]}]}`)
+
+	for _, tc := range []struct {
+		include []string
+		want    string
+	}{
+		{[]string{"title"}, "title,id"},
+		{[]string{"id", "title"}, "id,title"},
+	} {
+		if _, err := e.SearchPage(t.Context(), "posts", beachcomber.Query{Q: "x", IncludeFields: tc.include}); err != nil {
+			t.Fatal(err)
+		}
+		if got := onlyQuery(t, f).Get("include_fields"); got != tc.want {
+			t.Fatalf("include_fields for %v = %q, want %q", tc.include, got, tc.want)
+		}
+	}
+
+	res, err := e.SearchPage(t.Context(), "posts", beachcomber.Query{
+		Q: "blue", QueryBy: []string{"body"}, ExcludeFields: []string{"body"},
+		Highlight: &beachcomber.Highlight{Fields: []string{"body"}},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	params := onlyQuery(t, f)
+	if params.Get("exclude_fields") != "body" || params.Get("highlight_fields") != "body" || params.Has("include_fields") {
+		t.Fatalf("params = %v", params)
+	}
+	if len(res.Hits) != 1 || len(res.Hits[0].Highlights) != 1 || res.Hits[0].Highlights[0].Field != "body" {
+		t.Fatalf("an excluded field must still be highlighted: %+v", res.Hits)
+	}
+
+	for name, q := range map[string]beachcomber.Query{
+		"exclude_fields":      {ExcludeFields: []string{"body", "id"}},
+		"AffixTokens":         {Highlight: &beachcomber.Highlight{AffixTokens: -1}},
+		"MaxSnippetsPerArray": {Highlight: &beachcomber.Highlight{MaxSnippetsPerArray: -1}},
+	} {
+		_, err := e.SearchPage(t.Context(), "posts", q)
+		if err == nil || !strings.Contains(err.Error(), name) {
+			t.Fatalf("%s: err = %v", name, err)
+		}
+	}
+	if calls := f.take(); len(calls) != 0 {
+		t.Fatalf("a rejected query sent %d requests", len(calls))
+	}
+}
+
+func TestTypesenseSupportsHighlights(t *testing.T) {
+	_, e := newFake(t)
+	if !beachcomber.SupportsHighlights(e) {
+		t.Fatal("the Typesense engine supports highlights")
+	}
+	var _ beachcomber.HighlightSearcher = e
+}