feat: relevance-ordered search, suggest rows and journal:reindex

- Engine search asks Typesense for the first 250 candidates, re-gates them in SQL, counts the gated rows and pages over them in rank order; pages after the first are no longer empty. An explicit sort overrides the rank. - The settings row gates engine search, so it works straight after boot. - ?fields=suggest serves id, title, slug, a plain-text excerpt, featured_image_url and published_at. - Content is indexed as plain text; Post declares SearchIndexSchema. - journal:reindex [--fresh] syncs every post into the index.
2026-10-30 17:59:57 +01:00
parent 5b38a3b20d
commit 08e3d82ecd
9 changed files with 768 additions and 18 deletions
--- a/README.md
+++ b/README.md
@@ -22,6 +22,7 @@ WinterCMS theme components such as `journalPost` and `journalPosts` are a later
 - **Categories and tags.** `models.Category` is translatable (name, description, slug). It also stores `content`, an HTML host column that is not translatable, edited in the admin with the `blogeditor` field type. Fillable is name, slug, code, description, content and parent_id; nest_* columns exist in DDL and are not fillable. `models.Tag` is not translatable. Fillable is name, slug, description. Pivots are `golem15_journal_posts_categories` and `golem15_journal_posts_tags`.
 - **Menu and sitemap item types.** The plugin registers its five PHP menu item types (`journal-category`, `all-journal-categories`, `journal-post`, `all-journal-posts`, `category-journal-posts`) and `journal-tag` with the framework's menu item type registry, so menus and sitemaps can point at posts, categories and tags. Posts, categories and tags carry content descriptors, so plugins that work on every content type (sitemaps, SEO, search) describe, page and match Journal records without the Journal importing them. See [menu item types](#menu-item-types) and [content descriptors](#content-descriptors).
 - **Settings.** Singleton row ID=1 on `golem15_journal_settings`, not PHP `system_settings`. `search_use_typesense` defaults to false. A fresh save never contacts Typesense.
+- **Search.** With a configured search engine and `search_use_typesense` on, published posts are kept in the Typesense collection `golem15_journal_posts` after every committed save, and `GET /posts?search=` answers in relevance order. Otherwise search is a SQL `ILIKE`. `?fields=suggest` serves short rows for a search box, and `journal:reindex` fills the index. See [search](#search).
 - **Public HTTP.** Anonymous GET `/_journal/api/v1/posts`, `/posts/{slug}`, `/categories`, `/tags`, `/rss`, `/authors/{author}`, and `/authors/{author}/posts`. `/posts/{slug}` finds the post by its default-locale slug, then by its slug in the request locale (the router's locale, from `Accept-Language` or the translate plugin's resolver), then by id, so a translated slug resolves. Writes under the same prefix require a cabana backend JWT (`aud=backend`) or a personal admin API token (`g15_…`) whose scope covers the Journal.
 - **Record events.** Post writes through the admin fire the framework's record events (`pact.RecordSaved`, `pact.RecordDeleted`) after the commit, like every admin controller. The `/_journal/api/v1` post routes, which MCP tools call, fire the same events with `cabana.EmitRecordSaved` and `cabana.EmitRecordDeleted`: `POST /posts` a RecordSaved with `Created`, `PUT /posts/{id}` a RecordSaved whose changes hold the post's changed columns and each submitted translation that changed (a translated slug per locale), `DELETE /posts/{id}` a RecordDeleted. A tag `POST` or `PUT` creates from a name in `tags` fires its own RecordSaved with `Created`. `POST` and `PUT` write the row, its categories, tags and translations in one transaction and fire the event after it commits, so a write that fails part-way changes nothing and fires nothing. Their `Controller` is empty, as they do not go through an admin form. Another plugin (an SEO plugin creating redirects, a search indexer) therefore hears about a slug changed over MCP without importing the Journal.
 - **MCP tools.** `journal.Plugin.MCP` declares seven tools and one prompt for the framework's MCP endpoint (pier, a binary built with the `summercms_mcp` tag): `journal_list_posts`, `journal_show_post`, `journal_list_categories`, `journal_list_tags`, `journal_create_post`, `journal_update_post`, `journal_delete_post` and the prompt `journal_draft_post`. Each runs the matching `/_journal/api/v1` route as the backend account linked to the MCP user (guard `backend`) and is listed only when that account has `golem15.journal.access_posts`; the route checks its own permissions on every call. See [MCP tools](#mcp-tools).
@@ -142,6 +143,34 @@ Content-Type: application/json
 {"title":"Hello","content":"A blog post"}
 ```

+### Search
+
+`GET /_journal/api/v1/posts?search=<term>` filters the list by a term of 3 or more characters; a shorter term is ignored and the list comes back unfiltered.
+
+- **SQL.** By default, and whenever the engine cannot be used, the term is matched with `ILIKE` against title, slug, excerpt and content, and the rows keep the list order (`sort`, default newest first).
+- **Typesense.** When `search.driver` is `typesense` with an API key and the Journal setting `search_use_typesense` is on, the engine is asked for the first 250 candidates (`typesense.MaxPerPage`), ranked by text match with the setting weights for title, excerpt and content (default 5, 3, 1), then by date. The candidates are re-gated in SQL with the same filters as any list (published, `category`, `tag`, `author`), so a stale index entry never shows. `meta.total` counts the gated rows, the rows keep the engine's rank, with pinned posts first when `pinned_first` is set, and `page` and `per_page` page over them as over any list. An explicit `sort` overrides the rank. A search that matches more than 250 posts keeps the first 250. When the engine fails, the request is answered from SQL and the failure is logged at Warn.
+
+The setting row is read on every engine search, so turning `search_use_typesense` off takes effect on the next request, without a restart.
+
+`?fields=suggest` replaces each row with the short shape a search box needs, under the same pagination envelope, filters and limits:
+
+| Key | Value |
+|-----|-------|
+| `id` | Post id. |
+| `title` | Default-locale title, as in the full row. |
+| `slug` | Default-locale slug, as in the full row. |
+| `excerpt` | The excerpt as plain text, cut to about 160 characters on a word boundary with `…`; the content as plain text when the post has no excerpt. |
+| `featured_image_url` | The first featured image URL, or `null`. |
+| `published_at` | As in the full row. |
+
+Any other `fields` value, or none, serves the full rows.
+
+```http
+GET /_journal/api/v1/posts?search=borowik&fields=suggest&per_page=5
+```
+
+The search document holds the default-locale title, excerpt and content; content is indexed as plain text, from `content_html` (or `content` when nothing was rendered) with tags removed and entities decoded. The collection is created from `models.Post.SearchIndexSchema` when it does not exist. A collection created before the schema existed keeps its auto-detected fields until `journal:reindex --fresh` recreates it.
+
 ### Personal API tokens

 The plugin registers two scopes through `journal.Plugin.TokenScopes` (`pact.HasTokenScopes`):
@@ -333,7 +362,7 @@ The request is a `POST` to the URL with `Authorization: Bearer <token>`, `Conten
 | `journal.Plugin.LangFS` | Embedded English and Polish UI phrasebooks. |
 | `journal.Plugin.Migrations` | gormigrate steps registered by this plugin. |
 | `journal.Plugin.Models` | Models registered by this plugin. |
-| `journal.Plugin.Commands` | Registers `journal:export-posts` and `journal:import-posts`. |
+| `journal.Plugin.Commands` | Registers `journal:export-posts`, `journal:import-posts` and `journal:reindex`. |
 | `journal.Plugin.Routes` | Public GETs (posts, categories, tags, RSS, authors) and backend-Bearer writes under `/_journal/api/v1`. |
 | `journal.Plugin.MCP` | The MCP tools and the `draft_post` prompt (`pact.HasMCP`), guard `backend`, permission `golem15.journal.access_posts`. |
 | `journal.Plugin.TokenScopes` | Personal API token scopes `journal` and `journal:read` (`pact.HasTokenScopes`). |
@@ -346,7 +375,9 @@ The request is a `POST` to the URL with `Authorization: Bearer <token>`, `Conten
 | `models.Post.AttachRelations` | `featured_images` and `content_images` attachMany. |
 | `models.Post.SearchableAs` | Index name `golem15_journal_posts`. |
 | `models.Post.ShouldBeSearchable` | False when unpublished or `search_use_typesense` is off. |
-| `models.Post.ToSearchableArray` | Typesense document built after a committed save. |
+| `models.Post.ToSearchableArray` | Typesense document built after a committed save; content is plain text. |
+| `models.Post.SearchIndexSchema` | The Typesense collection schema a missing collection is created with (`beachcomber.IndexSchemaProvider`). |
+| `models.PlainText` | HTML to plain text: tags dropped, entities decoded, whitespace collapsed. |
 | `models.Post.CanEdit` | Owner or `golem15.journal.access_other_posts`. |
 | `models.Category` | Category row; `TableName` is `golem15_journal_categories`; `Content` holds the HTML body served as `content`. |
 | `models.Category.MorphName` | PHP class string `Golem15\Journal\Models\Category`. |
@@ -383,17 +414,19 @@ The request is a `POST` to the URL with `Authorization: Bearer <token>`, `Conten

 ## CLI commands

-Both commands write JSON files (one per post slug), matching the PHP `journal:export-posts` / `journal:import-posts` tools. They are not a CSV ImportExport framework. A blog application binary registers them through `Plugin.Commands`.
+The export and import commands write JSON files (one per post slug), matching the PHP `journal:export-posts` / `journal:import-posts` tools. They are not a CSV ImportExport framework. A blog application binary registers them through `Plugin.Commands`.

 | Command | Arguments and flags | Effect |
 |---------|---------------------|--------|
 | `journal:export-posts` | `--path` (default `journal-export`), `--dry-run` | Writes one JSON file per post, including non-default-locale translations, with a progress bar over the posts. Attachments are omitted. Ends with `Exported N posts to: <path>`. |
 | `journal:import-posts` | `--path` (default `journal-export`), `--force`, `--dry-run` | Creates posts from those files, one line per post. Existing slugs are skipped unless `--force` updates them. Ends with `Results: N created, N updated, N skipped, N failed`. |
+| `journal:reindex` | `--fresh` | Syncs every post into the search index in batches of 200: published posts are upserted, the others removed. `--fresh` first drops the collection and recreates it from the schema. Refuses, sending nothing, when the engine is not configured or `search_use_typesense` is off. Ends with `Reindexed: N synced, N removed, N failed` and exits non-zero on any failure. |

 ```sh
 ./bin/acme journal:export-posts --path journal-export
 ./bin/acme journal:import-posts --path journal-export --dry-run
 ./bin/acme journal:import-posts --path journal-export --force
+./bin/acme journal:reindex --fresh
 ```

 `./bin/acme help journal:import-posts` shows each command's long help and examples. The plugin logs through the component `journal` (`log.components.journal` in `config/log.yaml`).
--- a/console/logger.go
+++ b/console/logger.go
@@ -0,0 +1,7 @@
+package console
+
+import "git.golem15.com/golem15/summercms/modules/diary"
+
+// pluginLog is the plugin's logger: its records carry component=journal, so
+// config/log.yaml can raise or lower them (log.components.journal).
+var pluginLog = diary.For("journal")
--- a/console/reindex.go
+++ b/console/reindex.go
@@ -0,0 +1,141 @@
+package console
+
+import (
+	"context"
+	"errors"
+	"fmt"
+
+	"git.golem15.com/golem15/sm-journal-plugin/models"
+	"git.golem15.com/golem15/summercms/modules/backpack"
+	"git.golem15.com/golem15/summercms/modules/beachcomber"
+	"git.golem15.com/golem15/summercms/modules/bonfire"
+	"gorm.io/gorm"
+)
+
+// reindexBatch is how many posts one database read loads.
+const reindexBatch = 200
+
+// ReindexResult counts what a reindex did: posts upserted into the index,
+// posts removed from it (unpublished or scheduled) and posts whose sync
+// failed.
+type ReindexResult struct {
+	Synced  int
+	Removed int
+	Failed  int
+}
+
+// Reindex is journal:reindex.
+func Reindex(app *backpack.App) bonfire.Command {
+	return bonfire.Command{
+		Name:        "journal:reindex",
+		Description: "Sync every journal post into the search index",
+		Long: `Upserts the search document of every published post and removes the
+document of every unpublished or scheduled one, through the same path as the
+sync after a save. Posts written before search was turned on reach the index
+this way.
+
+--fresh first drops the collection and recreates it from the post schema
+(models.Post.SearchIndexSchema). Use it once on a collection that was
+auto-created without the schema, or after the schema changed; search answers
+from SQL while the collection is empty.
+
+The command refuses, and sends nothing, when the search engine is not
+configured (search.driver null, or no search.typesense.api_key) or the
+Journal setting search_use_typesense is off. It ends with
+"Reindexed: N synced, N removed, N failed" and exits non-zero when any post
+failed.`,
+		Flags: []bonfire.Flag{
+			{Name: "fresh", Description: "Drop and recreate the collection from the schema before syncing", Bare: true},
+		},
+		Examples: []bonfire.Example{
+			{Command: "journal:reindex", Description: "Sync every post into the existing collection"},
+			{Command: "journal:reindex --fresh", Description: "Recreate the collection from the schema, then sync every post"},
+		},
+		Run: func(ctx context.Context, in bonfire.Input, out bonfire.Output) error {
+			res, err := ReindexPosts(ctx, app, flagOn(in, "fresh"), out)
+			if err != nil {
+				return err
+			}
+			msg := fmt.Sprintf("Reindexed: %d synced, %d removed, %d failed", res.Synced, res.Removed, res.Failed)
+			if res.Failed > 0 {
+				out.Error(msg)
+				return fmt.Errorf("journal: %d posts failed to sync", res.Failed)
+			}
+			out.Success(msg)
+			return nil
+		},
+	}
+}
+
+// ReindexPosts syncs every post through beachcomber, in id batches. With
+// fresh it drops the collection and recreates it from the schema first. It
+// returns an error, before any request, when the engine is not configured
+// or search_use_typesense is off. A post that fails to sync is counted and
+// reported on out; the rest still run.
+func ReindexPosts(ctx context.Context, app *backpack.App, fresh bool, out bonfire.Output) (ReindexResult, error) {
+	var res ReindexResult
+	gdb := mustDB(app)
+	if gdb == nil {
+		return res, errors.New("journal: database is not configured")
+	}
+	svc, err := beachcomber.From(app)
+	if err != nil {
+		return res, err
+	}
+	engine := svc.Engine()
+	if engine == nil || !engine.Configured() {
+		return res, errors.New("journal: the search engine is not configured (set search.driver and search.typesense.api_key)")
+	}
+	var on bool
+	if err := gdb.WithContext(ctx).Raw(`SELECT search_use_typesense FROM golem15_journal_settings WHERE id = 1`).Scan(&on).Error; err != nil {
+		return res, fmt.Errorf("journal: read search_use_typesense: %w", err)
+	}
+	if !on {
+		return res, errors.New("journal: search is off; turn on search_use_typesense in the Journal settings first")
+	}
+	models.SetSearchGate(true)
+
+	index := svc.IndexName(&models.Post{})
+	schema := models.Post{}.SearchIndexSchema()
+	if fresh {
+		existed, err := beachcomber.DropIndex(ctx, engine, index)
+		if err != nil {
+			return res, fmt.Errorf("journal: drop %s: %w", index, err)
+		}
+		if existed {
+			line(out, "Dropped %s", index)
+		}
+	}
+	if err := beachcomber.EnsureIndex(ctx, engine, index, schema); err != nil {
+		return res, fmt.Errorf("journal: create %s: %w", index, err)
+	}
+
+	var batch []models.Post
+	err = gdb.WithContext(ctx).Model(&models.Post{}).
+		Select("id", "published", "published_at").
+		Order("id").
+		FindInBatches(&batch, reindexBatch, func(tx *gorm.DB, _ int) error {
+			for i := range batch {
+				post := &batch[i]
+				if err := svc.Sync(ctx, gdb, post); err != nil {
+					res.Failed++
+					if out != nil {
+						out.Warning(fmt.Sprintf("post %d: %v", post.ID, err))
+					}
+					continue
+				}
+				if post.ShouldBeSearchable() {
+					res.Synced++
+				} else {
+					res.Removed++
+				}
+			}
+			return ctx.Err()
+		}).Error
+	if err != nil {
+		return res, err
+	}
+	pluginLog.Info("journal: reindexed posts", "index", index, "fresh", fresh,
+		"synced", res.Synced, "removed", res.Removed, "failed", res.Failed)
+	return res, nil
+}
--- a/controllers/api/posts.go
+++ b/controllers/api/posts.go
@@ -26,10 +26,17 @@ import (
 const (
 	defaultPerPage = 9
 	maxPerPage     = 30
+
+	// fieldsSuggest is the ?fields value of the suggestion rows.
+	fieldsSuggest = "suggest"
+	// suggestExcerptRunes is about how long a suggestion's excerpt runs.
+	suggestExcerptRunes = 160
 )

 // Index is GET /_journal/api/v1/posts. Anonymous callers see published posts
 // only; an editor with access_posts may include drafts unless ?published=true.
+// With ?fields=suggest each row is the short suggestion shape
+// (serializeSuggestion).
 func Index(app *backpack.App) http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
 		r = optionalBackendPrincipal(app, r)
@@ -44,8 +51,11 @@ func Index(app *backpack.App) http.HandlerFunc {
 		perPage := clampPerPage(r.URL.Query().Get("per_page"))
 		page := max(1, atoiDefault(r.URL.Query().Get("page"), 1))

-		q := db.WithContext(r.Context()).Model(&models.Post{}).
-			Preload("User").Preload("Categories").Preload("Tags")
+		suggest := r.URL.Query().Get("fields") == fieldsSuggest
+		q := db.WithContext(r.Context()).Model(&models.Post{})
+		if !suggest {
+			q = q.Preload("User").Preload("Categories").Preload("Tags")
+		}

 		publishedOnly := true
 		if editor {
@@ -79,8 +89,11 @@ func Index(app *backpack.App) http.HandlerFunc {
 		if len([]rune(search)) < 3 {
 			search = ""
 		}
+		// rank is the engine's order of the candidates, nil for a SQL search.
+		var rank []uint
 		if search != "" {
-			if ids, used := searchWithEngine(app, r, db.WithContext(r.Context()), search, publishedOnly, page, perPage); used {
+			if ids, used := searchWithEngine(app, r, db.WithContext(r.Context()), search, publishedOnly); used {
+				rank = ids
 				if len(ids) == 0 {
 					q = q.Where("1 = 0")
 				} else {
@@ -102,7 +115,11 @@ func Index(app *backpack.App) http.HandlerFunc {
 		}

 		sort := strings.TrimSpace(r.URL.Query().Get("sort"))
-		order := listOrder(sort, queryBool(r.URL.Query().Get("pinned_first")))
+		pinnedFirst := queryBool(r.URL.Query().Get("pinned_first"))
+		order := listOrder(sort, pinnedFirst)
+		if len(rank) > 0 && sort == "" {
+			order = rankOrder(rank, pinnedFirst)
+		}
 		var posts []models.Post
 		if err := q.Order(order).Offset((page - 1) * perPage).Limit(perPage).Find(&posts).Error; err != nil {
 			writeAPIError(w, http.StatusInternalServerError, "Internal server error")
@@ -110,7 +127,11 @@ func Index(app *backpack.App) http.HandlerFunc {
 		}
 		rows := make([]map[string]any, 0, len(posts))
 		for i := range posts {
-			rows = append(rows, serializePost(&posts[i], db.WithContext(r.Context())))
+			if suggest {
+				rows = append(rows, serializeSuggestion(&posts[i], db.WithContext(r.Context())))
+			} else {
+				rows = append(rows, serializePost(&posts[i], db.WithContext(r.Context())))
+			}
 		}
 		wire.WriteJSON(w, http.StatusOK, lagoon.Paginate(rows, page, perPage, total))
 	}
@@ -585,10 +606,7 @@ func serializePost(p *models.Post, db *gorm.DB) map[string]any {
 		tags = append(tags, map[string]any{"id": t.ID, "name": t.Name, "slug": t.Slug})
 	}
 	featured := featuredImages(db, p)
-	var featuredURL any
-	if len(featured) > 0 {
-		featuredURL = featured[0]["url"]
-	}
+	featuredURL := firstImageURL(featured)
 	row := map[string]any{
 		"id":                 p.ID,
 		"title":              p.Title,
@@ -614,6 +632,57 @@ func serializePost(p *models.Post, db *gorm.DB) map[string]any {
 	return row
 }

+// serializeSuggestion is the ?fields=suggest row: a subset of serializePost's
+// keys with the same value formats, except that excerpt is plain text cut to
+// about suggestExcerptRunes on a word boundary, taken from the content when
+// the post has no excerpt.
+func serializeSuggestion(p *models.Post, db *gorm.DB) map[string]any {
+	if p == nil {
+		return map[string]any{}
+	}
+	return map[string]any{
+		"id":                 p.ID,
+		"title":              p.Title,
+		"slug":               p.Slug,
+		"excerpt":            suggestionExcerpt(p),
+		"featured_image_url": firstImageURL(featuredImages(db, p)),
+		"published_at":       carbonTime(p.PublishedAt),
+	}
+}
+
+func suggestionExcerpt(p *models.Post) string {
+	text := models.PlainText(p.Excerpt)
+	if text == "" {
+		body := p.ContentHTML
+		if strings.TrimSpace(body) == "" {
+			body = p.Content
+		}
+		text = models.PlainText(body)
+	}
+	return truncateWords(text, suggestExcerptRunes)
+}
+
+// truncateWords cuts s to at most n runes at the last space before the cut,
+// with an ellipsis; s is returned whole when it fits.
+func truncateWords(s string, n int) string {
+	runes := []rune(s)
+	if len(runes) <= n {
+		return s
+	}
+	cut := runes[:n]
+	if i := strings.LastIndexFunc(string(cut), unicode.IsSpace); i > 0 {
+		return strings.TrimRightFunc(string(cut)[:i], func(r rune) bool { return unicode.IsSpace(r) || unicode.IsPunct(r) }) + "…"
+	}
+	return string(cut) + "…"
+}
+
+func firstImageURL(featured []map[string]any) any {
+	if len(featured) == 0 {
+		return nil
+	}
+	return featured[0]["url"]
+}
+
 func serializeAuthor(user *cabana.BackendUser) any {
 	if user == nil {
 		return nil
--- a/controllers/api/posts_search.go
+++ b/controllers/api/posts_search.go
@@ -3,15 +3,27 @@ package api
 import (
 	"net/http"
 	"strconv"
+	"strings"

 	"git.golem15.com/golem15/sm-journal-plugin/models"
 	"git.golem15.com/golem15/summercms/modules/backpack"
 	"git.golem15.com/golem15/summercms/modules/beachcomber"
+	"git.golem15.com/golem15/summercms/modules/beachcomber/typesense"
 	"gorm.io/gorm"
 )

-func searchWithEngine(app *backpack.App, r *http.Request, db *gorm.DB, search string, publishedOnly bool, page, perPage int) ([]uint, bool) {
-	if app == nil || db == nil || !models.SearchGateEnabled() {
+// searchWindow is how many engine candidates one search re-gates in SQL.
+// The list pages over the gated rows, so a page never depends on how the
+// engine paged.
+const searchWindow = typesense.MaxPerPage
+
+// searchWithEngine returns the candidate post ids for search in the engine's
+// rank order, and whether the engine answered. It asks for the whole result
+// window (the first searchWindow hits), not one page: the caller re-gates
+// the ids in SQL, counts them and pages over them. When it reports false the
+// caller falls back to SQL search.
+func searchWithEngine(app *backpack.App, r *http.Request, db *gorm.DB, search string, publishedOnly bool) ([]uint, bool) {
+	if app == nil || db == nil {
 		return nil, false
 	}
 	svc, err := beachcomber.From(app)
@@ -20,17 +32,27 @@ func searchWithEngine(app *backpack.App, r *http.Request, db *gorm.DB, search st
 	}
 	var settings models.Settings
 	if err := db.First(&settings, 1).Error; err != nil {
+		if !models.SearchGateEnabled() {
+			return nil, false
+		}
 		settings.SearchTitleWeight = 5
 		settings.SearchExcerptWeight = 3
 		settings.SearchContentWeight = 1
+	} else {
+		// The row is the gate: the in-memory one is set only once a sync
+		// has run since boot.
+		models.SetSearchGate(settings.SearchUseTypesense)
+		if !settings.SearchUseTypesense {
+			return nil, false
+		}
 	}
 	q := beachcomber.Query{
 		Q:              search,
 		QueryBy:        []string{"title", "excerpt", "content"},
 		QueryByWeights: []int{settings.SearchTitleWeight, settings.SearchExcerptWeight, settings.SearchContentWeight},
 		SortBy:         "_text_match:desc,published_at:desc",
-		Page:           page,
-		PerPage:        perPage,
+		Page:           1,
+		PerPage:        searchWindow,
 	}
 	if publishedOnly {
 		q.FilterBy = "published:=1"
@@ -40,6 +62,9 @@ func searchWithEngine(app *backpack.App, r *http.Request, db *gorm.DB, search st
 		pluginLog.Warn("journal: Typesense search failed, falling back to DB search", "err", err)
 		return nil, false
 	}
+	if res.Found > searchWindow {
+		pluginLog.Debug("journal: search matched more posts than one window; keeping the first", "found", res.Found, "kept", searchWindow)
+	}
 	ids := make([]uint, 0, len(res.IDs))
 	for _, raw := range res.IDs {
 		n, err := strconv.ParseUint(raw, 10, 64)
@@ -50,3 +75,18 @@ func searchWithEngine(app *backpack.App, r *http.Request, db *gorm.DB, search st
 	}
 	return ids, true
 }
+
+// rankOrder orders rows by their position in ids, the engine's rank, with
+// pinned posts first when pinnedFirst. The ids are parsed integers, so the
+// array literal carries no caller text.
+func rankOrder(ids []uint, pinnedFirst bool) string {
+	parts := make([]string, len(ids))
+	for i, id := range ids {
+		parts[i] = strconv.FormatUint(uint64(id), 10)
+	}
+	order := "array_position(ARRAY[" + strings.Join(parts, ",") + "]::bigint[], id), id DESC"
+	if pinnedFirst {
+		return "is_pinned DESC, " + order
+	}
+	return order
+}
--- a/models/post_search.go
+++ b/models/post_search.go
@@ -2,14 +2,19 @@ package models

 import (
 	"context"
+	"html"
 	"strconv"
+	"strings"
 	"time"

 	"git.golem15.com/golem15/summercms/modules/beachcomber"
 	"gorm.io/gorm"
 )

-var _ beachcomber.Searchable = (*Post)(nil)
+var (
+	_ beachcomber.Searchable          = (*Post)(nil)
+	_ beachcomber.IndexSchemaProvider = (*Post)(nil)
+)

 // SearchableAs is the Typesense collection name (D-12).
 func (Post) SearchableAs() string { return "golem15_journal_posts" }
@@ -75,6 +80,54 @@ func (p *Post) ToSearchableArray(ctx context.Context, db *gorm.DB) (map[string]a
 		"tag_ids":      tagIDs,
 		"title":        p.Title,
 		"excerpt":      p.Excerpt,
-		"content":      p.Content,
+		"content":      PlainText(p.searchContent()),
 	}, nil
 }
+
+// searchContent is the post body to index: the rendered content_html, or
+// content when nothing has been rendered.
+func (p *Post) searchContent() string {
+	if strings.TrimSpace(p.ContentHTML) != "" {
+		return p.ContentHTML
+	}
+	return p.Content
+}
+
+// SearchIndexSchema is the Typesense collection schema. It applies only when
+// the collection is created; journal:reindex --fresh recreates it.
+func (Post) SearchIndexSchema() map[string]any {
+	return map[string]any{
+		"fields": []map[string]any{
+			{"name": "id", "type": "string"},
+			{"name": "title", "type": "string"},
+			{"name": "excerpt", "type": "string"},
+			{"name": "content", "type": "string"},
+			{"name": "published", "type": "int64"},
+			{"name": "published_at", "type": "int64"},
+			{"name": "is_pinned", "type": "int64"},
+			{"name": "user_id", "type": "int64"},
+			{"name": "created_at", "type": "int64"},
+			{"name": "category_ids", "type": "int64[]", "facet": false},
+			{"name": "tag_ids", "type": "int64[]", "facet": false},
+		},
+	}
+}
+
+// PlainText drops HTML tags and unescapes entities, with whitespace runs
+// collapsed to one space. A tag counts as a word break.
+func PlainText(s string) string {
+	var b strings.Builder
+	inTag := false
+	for _, r := range s {
+		switch {
+		case r == '<':
+			inTag = true
+			b.WriteRune(' ')
+		case r == '>':
+			inTag = false
+		case !inTag:
+			b.WriteRune(r)
+		}
+	}
+	return strings.Join(strings.Fields(html.UnescapeString(b.String())), " ")
+}
--- a/models/post_search_test.go
+++ b/models/post_search_test.go
@@ -0,0 +1,59 @@
+package models
+
+import (
+	"sort"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestSearchDocumentIsPlainText(t *testing.T) {
+	past := time.Now().Add(-time.Hour)
+	p := &Post{
+		ID:          7,
+		Title:       "Kurki",
+		Excerpt:     "Zajawka",
+		Content:     "## Kurki",
+		ContentHTML: `<h2 class="x">Kurki</h2><p>rosną&nbsp;w <a href="/las">lesie</a> &amp; pachną</p>`,
+		Published:   true,
+		PublishedAt: &past,
+	}
+	doc, err := p.ToSearchableArray(t.Context(), nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	content, _ := doc["content"].(string)
+	if strings.ContainsAny(content, "<>") || strings.Contains(content, "href") || strings.Contains(content, "&amp;") {
+		t.Fatalf("content keeps markup: %q", content)
+	}
+	if content != "Kurki rosną w lesie & pachną" {
+		t.Fatalf("content = %q", content)
+	}
+
+	p.ContentHTML = ""
+	doc, _ = p.ToSearchableArray(t.Context(), nil)
+	if doc["content"] != "## Kurki" {
+		t.Fatalf("content without content_html = %q", doc["content"])
+	}
+}
+
+// TestSearchSchemaMatchesDocument: the schema declares exactly the keys
+// ToSearchableArray emits.
+func TestSearchSchemaMatchesDocument(t *testing.T) {
+	doc, err := (&Post{ID: 1}).ToSearchableArray(t.Context(), nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	var docKeys, schemaKeys []string
+	for k := range doc {
+		docKeys = append(docKeys, k)
+	}
+	for _, f := range (Post{}).SearchIndexSchema()["fields"].([]map[string]any) {
+		schemaKeys = append(schemaKeys, f["name"].(string))
+	}
+	sort.Strings(docKeys)
+	sort.Strings(schemaKeys)
+	if strings.Join(docKeys, ",") != strings.Join(schemaKeys, ",") {
+		t.Fatalf("document keys %v, schema fields %v", docKeys, schemaKeys)
+	}
+}
--- a/plugin.go
+++ b/plugin.go
@@ -114,6 +114,7 @@ func (p *Plugin) Commands() []bonfire.Command {
 	return []bonfire.Command{
 		console.ExportPosts(p.app),
 		console.ImportPosts(p.app),
+		console.Reindex(p.app),
 	}
 }

--- a/search_engine_test.go
+++ b/search_engine_test.go
@@ -0,0 +1,347 @@
+package journal_test
+
+import (
+	"bufio"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+
+	"git.golem15.com/golem15/sm-journal-plugin/console"
+	"git.golem15.com/golem15/sm-journal-plugin/models"
+	"git.golem15.com/golem15/summercms/modules/bonfire"
+)
+
+const searchIndex = "golem15_journal_posts"
+
+// typesenseStub stands in for a Typesense server: one collection that GET,
+// POST and DELETE /collections create and drop, imports answered with one
+// success line per document, and a search that answers with hits, the ids
+// in rank order.
+type typesenseStub struct {
+	mu       sync.Mutex
+	exists   bool
+	hits     []string
+	requests []string
+	schema   map[string]any
+	imported []string
+	removed  []string
+}
+
+func (s *typesenseStub) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.requests = append(s.requests, r.Method+" "+r.URL.Path)
+	coll := "/collections/" + searchIndex
+	switch {
+	case r.Method == http.MethodGet && r.URL.Path == coll:
+		if !s.exists {
+			w.WriteHeader(http.StatusNotFound)
+			return
+		}
+		_, _ = io.WriteString(w, `{}`)
+	case r.Method == http.MethodPost && r.URL.Path == "/collections":
+		_ = json.NewDecoder(r.Body).Decode(&s.schema)
+		s.exists = true
+		w.WriteHeader(http.StatusCreated)
+		_, _ = io.WriteString(w, `{}`)
+	case r.Method == http.MethodDelete && r.URL.Path == coll:
+		if !s.exists {
+			w.WriteHeader(http.StatusNotFound)
+			return
+		}
+		s.exists = false
+		_, _ = io.WriteString(w, `{}`)
+	case r.Method == http.MethodPost && r.URL.Path == coll+"/documents/import":
+		sc := bufio.NewScanner(r.Body)
+		var answer []string
+		for sc.Scan() {
+			var doc map[string]any
+			if err := json.Unmarshal(sc.Bytes(), &doc); err == nil {
+				s.imported = append(s.imported, fmt.Sprint(doc["id"]))
+			}
+			answer = append(answer, `{"success":true}`)
+		}
+		_, _ = io.WriteString(w, strings.Join(answer, "\n"))
+	case r.Method == http.MethodDelete && strings.HasPrefix(r.URL.Path, coll+"/documents/"):
+		s.removed = append(s.removed, strings.TrimPrefix(r.URL.Path, coll+"/documents/"))
+		w.WriteHeader(http.StatusNotFound)
+	case r.Method == http.MethodGet && r.URL.Path == coll+"/documents/search":
+		hits := make([]string, len(s.hits))
+		for i, id := range s.hits {
+			hits[i] = `{"document":{"id":"` + id + `"}}`
+		}
+		_, _ = fmt.Fprintf(w, `{"found":%d,"hits":[%s]}`, len(s.hits), strings.Join(hits, ","))
+	default:
+		w.WriteHeader(http.StatusNotFound)
+	}
+}
+
+func (s *typesenseStub) sent() []string {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return append([]string(nil), s.requests...)
+}
+
+func newSearchEnv(t *testing.T) (*adminEnv, *typesenseStub) {
+	t.Helper()
+	stub := &typesenseStub{}
+	srv := httptest.NewServer(stub)
+	t.Cleanup(srv.Close)
+	u, err := url.Parse(srv.URL)
+	if err != nil {
+		t.Fatal(err)
+	}
+	env := newPostsEnv(t,
+		"SUMMER_SEARCH__DRIVER=typesense",
+		"SUMMER_SEARCH__TYPESENSE__API_KEY=test-only-typesense-key",
+		"SUMMER_SEARCH__TYPESENSE__HOST="+u.Hostname(),
+		"SUMMER_SEARCH__TYPESENSE__PORT="+u.Port(),
+		"SUMMER_SEARCH__TYPESENSE__PROTOCOL="+u.Scheme,
+	)
+	t.Cleanup(func() { models.SetSearchGate(false) })
+	return env, stub
+}
+
+func setSearchGate(t *testing.T, env *adminEnv, on bool) {
+	t.Helper()
+	if err := env.gdb.Exec(`UPDATE golem15_journal_settings SET search_use_typesense = ? WHERE id = 1`, on).Error; err != nil {
+		t.Fatal(err)
+	}
+}
+
+// seedSearchPosts creates n published posts titled "Grzyb N" and one draft,
+// with the gate off so the writes send nothing, and returns their ids.
+func seedSearchPosts(t *testing.T, env *adminEnv, n int) (published []uint, draft uint) {
+	t.Helper()
+	past := time.Now().UTC().Add(-time.Hour)
+	for i := 1; i <= n; i++ {
+		at := past.Add(-time.Duration(i) * time.Minute)
+		p := models.Post{
+			Title:       fmt.Sprintf("Grzyb %02d", i),
+			Slug:        fmt.Sprintf("grzyb-%02d", i),
+			Content:     "o grzybach",
+			Published:   true,
+			PublishedAt: &at,
+		}
+		if err := env.gdb.Create(&p).Error; err != nil {
+			t.Fatal(err)
+		}
+		published = append(published, p.ID)
+	}
+	d := models.Post{Title: "Grzyb szkic", Slug: "grzyb-szkic", Content: "o grzybach", Published: false}
+	if err := env.gdb.Create(&d).Error; err != nil {
+		t.Fatal(err)
+	}
+	return published, d.ID
+}
+
+type listBody struct {
+	Data []map[string]any `json:"data"`
+	Meta struct {
+		CurrentPage int   `json:"current_page"`
+		LastPage    int   `json:"last_page"`
+		Total       int64 `json:"total"`
+	} `json:"meta"`
+}
+
+func getList(t *testing.T, env *adminEnv, path string) listBody {
+	t.Helper()
+	rec := env.call(t, http.MethodGet, path, "", nil)
+	if rec.Code != http.StatusOK {
+		t.Fatalf("GET %s = %d: %s", path, rec.Code, rec.Body.String())
+	}
+	var body listBody
+	if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
+		t.Fatalf("GET %s body: %v", path, err)
+	}
+	return body
+}
+
+func slugsOf(rows []map[string]any) []string {
+	out := make([]string, len(rows))
+	for i, row := range rows {
+		out[i], _ = row["slug"].(string)
+	}
+	return out
+}
+
+// TestEngineSearchPages: the engine ranks 25 published posts and a draft;
+// the list re-gates them in SQL, keeps the engine's order across pages and
+// counts the gated rows only.
+func TestEngineSearchPages(t *testing.T) {
+	env, stub := newSearchEnv(t)
+	published, draft := seedSearchPosts(t, env, 25)
+	// Rank: oldest first (the reverse of the date order), the draft in the
+	// middle.
+	var rank []string
+	for i := len(published) - 1; i >= 0; i-- {
+		rank = append(rank, strconv.FormatUint(uint64(published[i]), 10))
+		if i == 12 {
+			rank = append(rank, strconv.FormatUint(uint64(draft), 10))
+		}
+	}
+	stub.hits = rank
+	setSearchGate(t, env, true)
+
+	want := make([]string, 0, 25)
+	for i := 25; i >= 1; i-- {
+		want = append(want, fmt.Sprintf("grzyb-%02d", i))
+	}
+	for page, slice := range map[int][]string{1: want[:10], 2: want[10:20], 3: want[20:]} {
+		body := getList(t, env, fmt.Sprintf("/_journal/api/v1/posts?search=grzyb&per_page=10&page=%d", page))
+		if body.Meta.Total != 25 {
+			t.Fatalf("page %d total = %d, want 25 (draft re-gated out)", page, body.Meta.Total)
+		}
+		if got := slugsOf(body.Data); strings.Join(got, ",") != strings.Join(slice, ",") {
+			t.Fatalf("page %d = %v, want %v", page, got, slice)
+		}
+	}
+	for _, req := range stub.sent() {
+		if strings.HasSuffix(req, "/documents/search") {
+			continue
+		}
+		t.Fatalf("unexpected Typesense request %q", req)
+	}
+
+	// An explicit sort overrides relevance.
+	body := getList(t, env, "/_journal/api/v1/posts?search=grzyb&per_page=3&sort=title+asc")
+	if got := strings.Join(slugsOf(body.Data), ","); got != "grzyb-01,grzyb-02,grzyb-03" {
+		t.Fatalf("sort=title asc = %s", got)
+	}
+}
+
+// TestEngineSearchGateOff: with Typesense configured but the gate off, the
+// list searches in SQL and sends nothing.
+func TestEngineSearchGateOff(t *testing.T) {
+	env, stub := newSearchEnv(t)
+	seedSearchPosts(t, env, 3)
+	stub.hits = []string{"999"}
+	body := getList(t, env, "/_journal/api/v1/posts?search=grzyb")
+	if body.Meta.Total != 3 || len(body.Data) != 3 {
+		t.Fatalf("total=%d rows=%d, want the 3 SQL matches", body.Meta.Total, len(body.Data))
+	}
+	if got := strings.Join(slugsOf(body.Data), ","); got != "grzyb-01,grzyb-02,grzyb-03" {
+		t.Fatalf("SQL order = %s, want the date order", got)
+	}
+	if n := len(stub.sent()); n != 0 {
+		t.Fatalf("Typesense requests = %v, want none", stub.sent())
+	}
+}
+
+// TestSuggestRows: ?fields=suggest serves only the suggestion keys, with a
+// plain-text excerpt cut on a word boundary or taken from the content.
+func TestSuggestRows(t *testing.T) {
+	env := newPostsEnv(t)
+	past := time.Now().UTC().Add(-time.Hour)
+	long := strings.Repeat("borowik szlachetny ", 20)
+	for _, p := range []models.Post{
+		{Title: "Z zajawką", Slug: "z-zajawka", Excerpt: "<p>Krótka <b>zajawka</b></p>", Content: "treść", Published: true, PublishedAt: &past},
+		{Title: "Długa", Slug: "dluga", Excerpt: long, Content: "treść", Published: true, PublishedAt: &past},
+		{Title: "Bez zajawki", Slug: "bez-zajawki", Content: "x", ContentHTML: "<h2>Kurki</h2><p>rosną &amp; pachną</p>", Published: true, PublishedAt: &past},
+	} {
+		if err := env.gdb.Create(&p).Error; err != nil {
+			t.Fatal(err)
+		}
+	}
+	body := getList(t, env, "/_journal/api/v1/posts?fields=suggest&per_page=5&sort=title+asc")
+	if len(body.Data) != 3 || body.Meta.Total != 3 {
+		t.Fatalf("rows=%d total=%d", len(body.Data), body.Meta.Total)
+	}
+	wantKeys := "excerpt,featured_image_url,id,published_at,slug,title"
+	excerpts := map[string]string{}
+	for _, row := range body.Data {
+		keys := make([]string, 0, len(row))
+		for k := range row {
+			keys = append(keys, k)
+		}
+		sort.Strings(keys)
+		if got := strings.Join(keys, ","); got != wantKeys {
+			t.Fatalf("suggest keys = %s, want %s", got, wantKeys)
+		}
+		excerpts[row["slug"].(string)] = row["excerpt"].(string)
+	}
+	if got := excerpts["z-zajawka"]; got != "Krótka zajawka" {
+		t.Fatalf("excerpt = %q", got)
+	}
+	if got := excerpts["bez-zajawki"]; got != "Kurki rosną & pachną" {
+		t.Fatalf("content fallback = %q", got)
+	}
+	got := excerpts["dluga"]
+	cut, ok := strings.CutSuffix(got, "…")
+	if !ok || len([]rune(cut)) > 160 || !strings.HasPrefix(long, cut+" ") {
+		t.Fatalf("long excerpt = %q, want under 160 runes cut on a word boundary", got)
+	}
+
+	// Without fields=suggest the rows are the full ones.
+	full := getList(t, env, "/_journal/api/v1/posts?per_page=1")
+	if _, ok := full.Data[0]["content"]; !ok {
+		t.Fatal("full rows lost content")
+	}
+}
+
+// TestReindexFresh: --fresh drops the collection, recreates it from the
+// schema, upserts the published posts and removes the draft's document.
+func TestReindexFresh(t *testing.T) {
+	env, stub := newSearchEnv(t)
+	published, draft := seedSearchPosts(t, env, 3)
+	stub.exists = true
+	setSearchGate(t, env, true)
+
+	var buf strings.Builder
+	out := bonfire.NewOutput(strings.NewReader(""), &buf, &buf)
+	res, err := console.ReindexPosts(t.Context(), env.app, true, out)
+	if err != nil {
+		t.Fatalf("reindex: %v (%s)", err, buf.String())
+	}
+	if res.Synced != 3 || res.Removed != 1 || res.Failed != 0 {
+		t.Fatalf("result = %+v, want 3 synced, 1 removed", res)
+	}
+	sent := stub.sent()
+	if len(sent) < 3 || sent[0] != "DELETE /collections/"+searchIndex ||
+		sent[1] != "GET /collections/"+searchIndex || sent[2] != "POST /collections" {
+		t.Fatalf("requests = %v, want DELETE, GET, POST /collections first", sent)
+	}
+	fields, _ := stub.schema["fields"].([]any)
+	if stub.schema["name"] != searchIndex || len(fields) != len(models.Post{}.SearchIndexSchema()["fields"].([]map[string]any)) {
+		t.Fatalf("created schema = %v", stub.schema)
+	}
+	if len(stub.imported) != 3 {
+		t.Fatalf("imported = %v, want the 3 published posts", stub.imported)
+	}
+	for i, id := range published {
+		if stub.imported[i] != strconv.FormatUint(uint64(id), 10) {
+			t.Fatalf("imported = %v, want %v", stub.imported, published)
+		}
+	}
+	if len(stub.removed) != 1 || stub.removed[0] != strconv.FormatUint(uint64(draft), 10) {
+		t.Fatalf("removed = %v, want the draft %d", stub.removed, draft)
+	}
+}
+
+// TestReindexRefuses: with the gate off, or no engine configured, the
+// command errors and sends nothing.
+func TestReindexRefuses(t *testing.T) {
+	env, stub := newSearchEnv(t)
+	seedSearchPosts(t, env, 1)
+	if _, err := console.ReindexPosts(t.Context(), env.app, true, nil); err == nil || !strings.Contains(err.Error(), "search_use_typesense") {
+		t.Fatalf("gate off: err = %v", err)
+	}
+	if n := len(stub.sent()); n != 0 {
+		t.Fatalf("gate off sent %v", stub.sent())
+	}
+
+	plain := newPostsEnv(t)
+	setSearchGate(t, plain, true)
+	if _, err := console.ReindexPosts(t.Context(), plain.app, false, nil); err == nil || !strings.Contains(err.Error(), "not configured") {
+		t.Fatalf("null driver: err = %v", err)
+	}
+}