feat(15-03): add journal media upload, RSS, and off-by-default Typesense gate
- POST /_journal/api/v1/media/upload requires access_posts and stays under journal/ - Anonymous GET rss is RSS 2.0 honoring rss_* settings; unpublished posts are omitted - beachcomber Gate reads search_use_typesense (default false) so a fresh save makes no Typesense HTTP Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
33
README.md
33
README.md
@@ -20,7 +20,8 @@ WinterCMS theme components such as `journalPost` and `journalPosts` are a later
|
||||
|
||||
- **Posts.** `models.Post` maps `golem15_journal_posts`. Translatable fields are title, content, content_html, excerpt, metadata, and slug (indexed). Fillable does not include `redactor_id`, `content_html`, or `user_id`. Attachments `featured_images` and `content_images` use `system_files` under the same MorphName. YAML admin uses `mltext`/`mlmarkdown`. `classes.FormatHTML` regenerates `content_html` on admin save.
|
||||
- **Categories and tags.** `models.Category` is translatable (name, description, slug). Fillable is name, slug, code, description, parent_id; nest_* columns exist in DDL and are not fillable. `models.Tag` is not translatable. Fillable is name, slug, description. Pivots are `golem15_journal_posts_categories` and `golem15_journal_posts_tags`.
|
||||
- **Settings.** Singleton row ID=1 on `golem15_journal_settings`, not PHP `system_settings`. `search_use_typesense` defaults to false.
|
||||
- **Settings.** Singleton row ID=1 on `golem15_journal_settings`, not PHP `system_settings`. `search_use_typesense` defaults to false. A fresh save never contacts Typesense.
|
||||
- **Public HTTP.** Anonymous GET `/_journal/api/v1/posts`, `/posts/{slug}`, `/categories`, `/tags`, and `/rss`. Writes under the same prefix require a cabana backend JWT (`aud=backend`).
|
||||
- **Author slug.** Plugin migration adds `backend_users.golem15_bloghub_author_slug`.
|
||||
- **Import and export.** `journal:export-posts` and `journal:import-posts` write and read one JSON file per post. The Posts list toolbar exposes the same actions to administrators with `golem15.journal.access_import_export`.
|
||||
- **Phrasebooks.** Embedded `lang/en/lang.yaml` and `lang/pl/lang.yaml` hold plugin, journal, post, category, and tag UI strings. They are independent of model translation JSON.
|
||||
@@ -67,6 +68,26 @@ The plugin's own `go.mod` replaces the framework with `../summercms.go` for a si
|
||||
|
||||
`Requires()` is exactly `golem15.translate`.
|
||||
|
||||
A blog frontend lists published posts and fetches one by slug without a token:
|
||||
|
||||
```http
|
||||
GET /_journal/api/v1/posts
|
||||
GET /_journal/api/v1/posts/hello-world
|
||||
GET /_journal/api/v1/categories
|
||||
GET /_journal/api/v1/tags
|
||||
GET /_journal/api/v1/rss
|
||||
```
|
||||
|
||||
Editors create a post with a cabana backend JWT (audience `backend`):
|
||||
|
||||
```http
|
||||
POST /_journal/api/v1/posts
|
||||
Authorization: Bearer <backend-access-token>
|
||||
Content-Type: application/json
|
||||
|
||||
{"title":"Hello","content":"A blog post"}
|
||||
```
|
||||
|
||||
A blog post model already declares translatable fields:
|
||||
|
||||
```go
|
||||
@@ -87,10 +108,16 @@ func (Post) MorphName() string { return `Golem15\Journal\Models\Post
|
||||
| `journal.Plugin.Migrations` | gormigrate steps registered by this plugin. |
|
||||
| `journal.Plugin.Models` | Models registered by this plugin. |
|
||||
| `journal.Plugin.Commands` | Registers `journal:export-posts` and `journal:import-posts`. |
|
||||
| `journal.Plugin.Routes` | Public GETs and backend-Bearer writes under `/_journal/api/v1`. |
|
||||
| `journal.Plugin.Buckets` | Limiter buckets `journal-public-api` and `journal-api` (Max 120). |
|
||||
| `models.Post` | Post row; `TableName` is `golem15_journal_posts`. |
|
||||
| `models.Post.MorphName` | PHP class string `Golem15\Journal\Models\Post`. |
|
||||
| `models.Post.Fillable` | Allow-list excluding `redactor_id`, `content_html`, and `user_id`. |
|
||||
| `models.Post.AttachRelations` | `featured_images` and `content_images` attachMany. |
|
||||
| `models.Post.SearchableAs` | Index name `golem15_journal_posts`. |
|
||||
| `models.Post.ShouldBeSearchable` | False when unpublished or `search_use_typesense` is off. |
|
||||
| `models.Post.ToSearchableArray` | Typesense document built after a committed save. |
|
||||
| `models.Post.CanEdit` | Owner or `golem15.journal.access_other_posts`. |
|
||||
| `models.Category` | Category row; `TableName` is `golem15_journal_categories`. |
|
||||
| `models.Category.MorphName` | PHP class string `Golem15\Journal\Models\Category`. |
|
||||
| `models.Category.Fillable` | `name`, `slug`, `code`, `description`, `parent_id`. |
|
||||
@@ -117,9 +144,9 @@ Both commands write JSON files (one per post slug), matching the PHP `journal:ex
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: backpack, bonfire, cabana, lagoon/attach, pact, party.
|
||||
- SummerCMS modules: backpack, beachcomber, bonfire, cabana, lagoon/attach, pact, party, surf.
|
||||
- Compiled plugin: `golem15.translate`.
|
||||
- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `github.com/yuin/goldmark`. Test-only: `github.com/testcontainers/testcontainers-go`.
|
||||
- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `github.com/yuin/goldmark`, `gocloud.dev`. Test-only: `github.com/testcontainers/testcontainers-go`.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
147
controllers/api/media.go
Normal file
147
controllers/api/media.go
Normal file
@@ -0,0 +1,147 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"io"
|
||||
"net/http"
|
||||
"path"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/lagoon/attach"
|
||||
"git.golem15.com/golem15/summercms/modules/wire"
|
||||
"gocloud.dev/blob"
|
||||
"gocloud.dev/gcerrors"
|
||||
)
|
||||
|
||||
const mediaMaxBytes = 10240 << 10
|
||||
|
||||
var mediaFolderPattern = regexp.MustCompile(`^[A-Za-z0-9_\-/]*$`)
|
||||
|
||||
// UploadMedia is POST /_journal/api/v1/media/upload (JOURNAL-006).
|
||||
func UploadMedia(app *backpack.App) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
r, pr, ok := requireBackendPrincipal(app, w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if !requireAccessPosts(w, pr) {
|
||||
return
|
||||
}
|
||||
folder := strings.TrimSpace(r.FormValue("folder"))
|
||||
if len(folder) > 100 || !mediaFolderPattern.MatchString(folder) || strings.Contains(folder, "..") {
|
||||
writeValidation(w, map[string][]string{"folder": {"The folder format is invalid."}})
|
||||
return
|
||||
}
|
||||
file, header, err := r.FormFile("file")
|
||||
if err != nil {
|
||||
writeValidation(w, map[string][]string{"file": {"The file field is required."}})
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
name := path.Base(strings.ReplaceAll(header.Filename, `\`, "/"))
|
||||
ext := strings.ToLower(strings.TrimPrefix(path.Ext(name), "."))
|
||||
switch ext {
|
||||
case "jpg", "jpeg", "png", "gif", "webp":
|
||||
default:
|
||||
writeValidation(w, map[string][]string{"file": {"The file must be an image."}})
|
||||
return
|
||||
}
|
||||
bucket, ok := app.Lookup[*blob.Bucket]()
|
||||
if !ok || bucket == nil {
|
||||
writeAPIError(w, http.StatusInternalServerError, "Internal server error")
|
||||
return
|
||||
}
|
||||
keyPath := mediaObjectPath(folder, name)
|
||||
body, err := io.ReadAll(io.LimitReader(file, mediaMaxBytes+1))
|
||||
if err != nil {
|
||||
writeAPIError(w, http.StatusInternalServerError, "Internal server error")
|
||||
return
|
||||
}
|
||||
if int64(len(body)) > mediaMaxBytes {
|
||||
writeValidation(w, map[string][]string{"file": {"The file may not be greater than 10240 kilobytes."}})
|
||||
return
|
||||
}
|
||||
keyPath, err = uniqueMediaKey(r, bucket, keyPath)
|
||||
if err != nil {
|
||||
writeAPIError(w, http.StatusInternalServerError, "Internal server error")
|
||||
return
|
||||
}
|
||||
wtr, err := bucket.NewWriter(r.Context(), strings.TrimPrefix(keyPath, "/"), nil)
|
||||
if err != nil {
|
||||
writeAPIError(w, http.StatusInternalServerError, "Internal server error")
|
||||
return
|
||||
}
|
||||
if _, err := wtr.Write(body); err != nil {
|
||||
_ = wtr.Close()
|
||||
writeAPIError(w, http.StatusInternalServerError, "Internal server error")
|
||||
return
|
||||
}
|
||||
if err := wtr.Close(); err != nil {
|
||||
writeAPIError(w, http.StatusInternalServerError, "Internal server error")
|
||||
return
|
||||
}
|
||||
wire.WriteJSON(w, http.StatusCreated, map[string]any{
|
||||
"data": map[string]any{
|
||||
"url": attach.PublicURL(strings.TrimPrefix(keyPath, "/")),
|
||||
"path": keyPath,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func mediaObjectPath(folder, fileName string) string {
|
||||
segments := make([]string, 0, 4)
|
||||
for _, part := range strings.Split(strings.Trim(folder, "/"), "/") {
|
||||
if part == "" || part == "." || part == ".." {
|
||||
continue
|
||||
}
|
||||
segments = append(segments, part)
|
||||
}
|
||||
if len(segments) > 0 && strings.EqualFold(segments[0], "journal") {
|
||||
segments = segments[1:]
|
||||
}
|
||||
folder = strings.Join(segments, "/")
|
||||
if folder == "" {
|
||||
return "/journal/" + fileName
|
||||
}
|
||||
return "/journal/" + folder + "/" + fileName
|
||||
}
|
||||
|
||||
func uniqueMediaKey(r *http.Request, bucket *blob.Bucket, pathName string) (string, error) {
|
||||
key := strings.TrimPrefix(pathName, "/")
|
||||
ok, err := bucket.Exists(r.Context(), key)
|
||||
if err != nil && gcerrors.Code(err) != gcerrors.NotFound {
|
||||
return "", err
|
||||
}
|
||||
if !ok {
|
||||
return pathName, nil
|
||||
}
|
||||
ext := path.Ext(pathName)
|
||||
base := strings.TrimSuffix(pathName, ext)
|
||||
for n := 1; n < 1000; n++ {
|
||||
candidate := base + "-" + itoa(n) + ext
|
||||
exists, err := bucket.Exists(r.Context(), strings.TrimPrefix(candidate, "/"))
|
||||
if err != nil && gcerrors.Code(err) != gcerrors.NotFound {
|
||||
return "", err
|
||||
}
|
||||
if !exists {
|
||||
return candidate, nil
|
||||
}
|
||||
}
|
||||
return pathName, nil
|
||||
}
|
||||
|
||||
func itoa(n int) string {
|
||||
if n == 0 {
|
||||
return "0"
|
||||
}
|
||||
var b [16]byte
|
||||
i := len(b)
|
||||
for n > 0 {
|
||||
i--
|
||||
b[i] = byte('0' + n%10)
|
||||
n /= 10
|
||||
}
|
||||
return string(b[i:])
|
||||
}
|
||||
@@ -77,11 +77,19 @@ func Index(app *backpack.App) http.HandlerFunc {
|
||||
search = ""
|
||||
}
|
||||
if search != "" {
|
||||
like := "%" + escapeLike(search) + "%"
|
||||
q = q.Where(
|
||||
"(title ILIKE ? ESCAPE '\\' OR slug ILIKE ? ESCAPE '\\' OR excerpt ILIKE ? ESCAPE '\\' OR content ILIKE ? ESCAPE '\\')",
|
||||
like, like, like, like,
|
||||
)
|
||||
if ids, used := searchWithEngine(app, r, db.WithContext(r.Context()), search, publishedOnly, page, perPage); used {
|
||||
if len(ids) == 0 {
|
||||
q = q.Where("1 = 0")
|
||||
} else {
|
||||
q = q.Where("id IN ?", ids)
|
||||
}
|
||||
} else {
|
||||
like := "%" + escapeLike(search) + "%"
|
||||
q = q.Where(
|
||||
"(title ILIKE ? ESCAPE '\\' OR slug ILIKE ? ESCAPE '\\' OR excerpt ILIKE ? ESCAPE '\\' OR content ILIKE ? ESCAPE '\\')",
|
||||
like, like, like, like,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
var total int64
|
||||
@@ -181,15 +189,6 @@ func Tags(app *backpack.App) http.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// RSS is GET /_journal/api/v1/rss. Task 3 fills the PHP RSS contract.
|
||||
func RSS(app *backpack.App) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/rss+xml; charset=utf-8")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(`<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"><channel></channel></rss>`))
|
||||
}
|
||||
}
|
||||
|
||||
// Store is POST /_journal/api/v1/posts. Field-by-field create; never lagoon.Fill.
|
||||
func Store(app *backpack.App) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
53
controllers/api/posts_search.go
Normal file
53
controllers/api/posts_search.go
Normal file
@@ -0,0 +1,53 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
"git.golem15.com/golem15/sm-journal-plugin/models"
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/beachcomber"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func searchWithEngine(app *backpack.App, r *http.Request, db *gorm.DB, search string, publishedOnly bool, page, perPage int) ([]uint, bool) {
|
||||
if app == nil || db == nil || !models.SearchGateEnabled() {
|
||||
return nil, false
|
||||
}
|
||||
svc, err := beachcomber.From(app)
|
||||
if err != nil || svc == nil || svc.Engine() == nil || !svc.Engine().Configured() {
|
||||
return nil, false
|
||||
}
|
||||
var settings models.Settings
|
||||
if err := db.First(&settings, 1).Error; err != nil {
|
||||
settings.SearchTitleWeight = 5
|
||||
settings.SearchExcerptWeight = 3
|
||||
settings.SearchContentWeight = 1
|
||||
}
|
||||
q := beachcomber.Query{
|
||||
Q: search,
|
||||
QueryBy: []string{"title", "excerpt", "content"},
|
||||
QueryByWeights: []int{settings.SearchTitleWeight, settings.SearchExcerptWeight, settings.SearchContentWeight},
|
||||
SortBy: "_text_match:desc,published_at:desc",
|
||||
Page: page,
|
||||
PerPage: perPage,
|
||||
}
|
||||
if publishedOnly {
|
||||
q.FilterBy = "published:=1"
|
||||
}
|
||||
res, err := beachcomber.SearchPage(r.Context(), svc.Engine(), svc.IndexName(&models.Post{}), q)
|
||||
if err != nil {
|
||||
slog.Default().Warn("journal: Typesense search failed, falling back to DB search", "err", err)
|
||||
return nil, false
|
||||
}
|
||||
ids := make([]uint, 0, len(res.IDs))
|
||||
for _, raw := range res.IDs {
|
||||
n, err := strconv.ParseUint(raw, 10, 64)
|
||||
if err != nil || n == 0 {
|
||||
continue
|
||||
}
|
||||
ids = append(ids, uint(n))
|
||||
}
|
||||
return ids, true
|
||||
}
|
||||
144
controllers/api/rss.go
Normal file
144
controllers/api/rss.go
Normal file
@@ -0,0 +1,144 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/xml"
|
||||
"html"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.golem15.com/golem15/sm-journal-plugin/models"
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
)
|
||||
|
||||
// RSS is GET /_journal/api/v1/rss. Empty or disabled settings still yield a
|
||||
// well-formed RSS 2.0 document (D-14).
|
||||
func RSS(app *backpack.App) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
db, err := gdb(app)
|
||||
if err != nil {
|
||||
http.Error(w, "Internal server error", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
var settings models.Settings
|
||||
if err := db.WithContext(r.Context()).First(&settings, 1).Error; err != nil {
|
||||
settings = models.Settings{RSSLanguage: "en-us", RSSPostsPerFeed: 20, RSSEnabled: true}
|
||||
}
|
||||
limit := settings.RSSPostsPerFeed
|
||||
if limit <= 0 {
|
||||
limit = 20
|
||||
}
|
||||
var posts []models.Post
|
||||
if err := applyPublished(db.WithContext(r.Context()).Model(&models.Post{})).
|
||||
Preload("User").Preload("Categories").
|
||||
Order("published_at DESC").Limit(limit).Find(&posts).Error; err != nil {
|
||||
http.Error(w, "Internal server error", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
title := "Journal"
|
||||
if settings.RSSTitle != nil && strings.TrimSpace(*settings.RSSTitle) != "" {
|
||||
title = strings.TrimSpace(*settings.RSSTitle)
|
||||
}
|
||||
desc := ""
|
||||
if settings.RSSDescription != nil {
|
||||
desc = *settings.RSSDescription
|
||||
}
|
||||
site := requestOrigin(r, app)
|
||||
self := site + r.URL.RequestURI()
|
||||
var b strings.Builder
|
||||
b.WriteString(`<?xml version="1.0" encoding="utf-8"?>` + "\n")
|
||||
b.WriteString(`<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:media="http://search.yahoo.com/mrss/">` + "\n")
|
||||
b.WriteString("<channel>\n")
|
||||
writeXMLTag(&b, "title", title)
|
||||
writeXMLTag(&b, "link", site+"/")
|
||||
writeXMLTag(&b, "description", desc)
|
||||
if settings.RSSLanguage != "" {
|
||||
writeXMLTag(&b, "language", settings.RSSLanguage)
|
||||
}
|
||||
if settings.RSSCopyright != nil && strings.TrimSpace(*settings.RSSCopyright) != "" {
|
||||
writeXMLTag(&b, "copyright", *settings.RSSCopyright)
|
||||
}
|
||||
writeXMLTag(&b, "lastBuildDate", time.Now().UTC().Format(time.RFC1123Z))
|
||||
writeXMLTag(&b, "generator", "Golem15 Journal")
|
||||
b.WriteString(`<atom:link rel="self" href="` + xmlEsc(self) + `" type="application/rss+xml" />` + "\n")
|
||||
if settings.RSSImageURL != nil && strings.TrimSpace(*settings.RSSImageURL) != "" {
|
||||
b.WriteString("<image>\n")
|
||||
writeXMLTag(&b, "url", *settings.RSSImageURL)
|
||||
writeXMLTag(&b, "title", title)
|
||||
writeXMLTag(&b, "link", site+"/")
|
||||
b.WriteString("</image>\n")
|
||||
}
|
||||
for i := range posts {
|
||||
p := &posts[i]
|
||||
postURL := site + "/blog/" + p.Slug
|
||||
summary := p.Excerpt
|
||||
if strings.TrimSpace(summary) == "" {
|
||||
summary = truncateRunes(stripTags(p.ContentHTML), 300)
|
||||
}
|
||||
b.WriteString("<item>\n")
|
||||
writeXMLTag(&b, "title", p.Title)
|
||||
writeXMLTag(&b, "link", postURL)
|
||||
b.WriteString(`<guid isPermaLink="true">` + xmlEsc(postURL) + "</guid>\n")
|
||||
if p.PublishedAt != nil {
|
||||
writeXMLTag(&b, "pubDate", p.PublishedAt.UTC().Format(time.RFC1123Z))
|
||||
}
|
||||
b.WriteString("<description><![CDATA[" + cdata(summary) + "]]></description>\n")
|
||||
if settings.RSSIncludeContent && p.ContentHTML != "" {
|
||||
b.WriteString("<content:encoded><![CDATA[" + cdata(p.ContentHTML) + "]]></content:encoded>\n")
|
||||
}
|
||||
if p.User != nil {
|
||||
creator := strings.TrimSpace(p.User.FirstName + " " + p.User.LastName)
|
||||
if creator != "" {
|
||||
b.WriteString("<dc:creator>" + xmlEsc(creator) + "</dc:creator>\n")
|
||||
}
|
||||
}
|
||||
for _, cat := range p.Categories {
|
||||
writeXMLTag(&b, "category", cat.Name)
|
||||
}
|
||||
b.WriteString("</item>\n")
|
||||
}
|
||||
b.WriteString("</channel>\n</rss>\n")
|
||||
w.Header().Set("Content-Type", "application/rss+xml; charset=utf-8")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(b.String()))
|
||||
}
|
||||
}
|
||||
|
||||
func writeXMLTag(b *strings.Builder, name, value string) {
|
||||
b.WriteString("<" + name + ">" + xmlEsc(value) + "</" + name + ">\n")
|
||||
}
|
||||
|
||||
func xmlEsc(s string) string {
|
||||
var b strings.Builder
|
||||
_ = xml.EscapeText(&b, []byte(s))
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func cdata(s string) string {
|
||||
return strings.ReplaceAll(s, "]]>", "]]>")
|
||||
}
|
||||
|
||||
func requestOrigin(r *http.Request, app *backpack.App) string {
|
||||
if app != nil && app.Config != nil {
|
||||
if u := strings.TrimRight(strings.TrimSpace(app.Config.String("url")), "/"); u != "" {
|
||||
return u
|
||||
}
|
||||
}
|
||||
scheme := "http"
|
||||
if r.TLS != nil {
|
||||
scheme = "https"
|
||||
}
|
||||
return scheme + "://" + r.Host
|
||||
}
|
||||
|
||||
func truncateRunes(s string, n int) string {
|
||||
if utf8.RuneCountInString(s) <= n {
|
||||
return s
|
||||
}
|
||||
runes := []rune(html.UnescapeString(s))
|
||||
if len(runes) > n {
|
||||
return string(runes[:n])
|
||||
}
|
||||
return s
|
||||
}
|
||||
226
journal_api_task3_test.go
Normal file
226
journal_api_task3_test.go
Normal file
@@ -0,0 +1,226 @@
|
||||
package journal_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/xml"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/sm-journal-plugin/controllers"
|
||||
"git.golem15.com/golem15/sm-journal-plugin/models"
|
||||
)
|
||||
|
||||
var png1x1 = []byte{
|
||||
0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
|
||||
0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
|
||||
0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01,
|
||||
0x08, 0x06, 0x00, 0x00, 0x00, 0x1f, 0x15, 0xc4,
|
||||
0x89, 0x00, 0x00, 0x00, 0x0a, 0x49, 0x44, 0x41, 0x54,
|
||||
0x78, 0x9c, 0x63, 0x00, 0x01, 0x00, 0x00, 0x05, 0x00, 0x01,
|
||||
0x0d, 0x0a, 0x2d, 0xb4, 0x00, 0x00, 0x00, 0x00,
|
||||
0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82,
|
||||
}
|
||||
|
||||
func (e *adminEnv) callMultipart(t *testing.T, method, path, token string, fields map[string]string, fileField, fileName string, fileBody []byte) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
var buf bytes.Buffer
|
||||
mw := multipart.NewWriter(&buf)
|
||||
for k, v := range fields {
|
||||
if err := mw.WriteField(k, v); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if fileField != "" {
|
||||
part, err := mw.CreateFormFile(fileField, fileName)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := part.Write(fileBody); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := mw.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
req := httptest.NewRequest(method, path, &buf)
|
||||
req.Host = "example.com"
|
||||
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||
if token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
e.h.ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
func TestJournal006MediaUpload(t *testing.T) {
|
||||
env := newPostsEnv(t)
|
||||
noPerm := env.admin(t)
|
||||
denied := env.callMultipart(t, http.MethodPost, "/_journal/api/v1/media/upload", noPerm, nil, "file", "cover.png", png1x1)
|
||||
if denied.Code != http.StatusForbidden {
|
||||
t.Fatalf("upload without access_posts = %d, want 403: %s", denied.Code, denied.Body.String())
|
||||
}
|
||||
if got, _ := phpErrorString(t, denied.Body.Bytes()); got != "Insufficient permissions" {
|
||||
t.Fatalf("without access_posts error = %q body=%s", got, denied.Body.String())
|
||||
}
|
||||
|
||||
okTok := env.admin(t, controllers.PermissionAccessPosts)
|
||||
created := env.callMultipart(t, http.MethodPost, "/_journal/api/v1/media/upload", okTok, nil, "file", "cover.png", png1x1)
|
||||
if created.Code != http.StatusCreated {
|
||||
t.Fatalf("upload with access_posts = %d, want 201: %s", created.Code, created.Body.String())
|
||||
}
|
||||
var body struct {
|
||||
Data struct {
|
||||
URL string `json:"url"`
|
||||
Path string `json:"path"`
|
||||
} `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(created.Body.Bytes(), &body); err != nil {
|
||||
t.Fatalf("upload json: %s err=%v", created.Body.String(), err)
|
||||
}
|
||||
if !strings.HasPrefix(body.Data.Path, "/journal/") {
|
||||
t.Fatalf("path = %q, want prefix /journal/", body.Data.Path)
|
||||
}
|
||||
|
||||
trav := env.callMultipart(t, http.MethodPost, "/_journal/api/v1/media/upload", okTok, map[string]string{"folder": "../etc"}, "file", "cover.png", png1x1)
|
||||
if trav.Code != http.StatusUnprocessableEntity {
|
||||
t.Fatalf("folder traversal = %d, want 422: %s", trav.Code, trav.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchGateOff(t *testing.T) {
|
||||
var hits atomic.Int64
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
hits.Add(1)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = io.WriteString(w, `{}`)
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
u, err := url.Parse(srv.URL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
env := newPostsEnv(t,
|
||||
"SUMMER_SEARCH__DRIVER=typesense",
|
||||
"SUMMER_SEARCH__TYPESENSE__API_KEY=test-only-typesense-key",
|
||||
"SUMMER_SEARCH__TYPESENSE__HOST="+u.Hostname(),
|
||||
"SUMMER_SEARCH__TYPESENSE__PORT="+u.Port(),
|
||||
"SUMMER_SEARCH__TYPESENSE__PROTOCOL="+u.Scheme,
|
||||
)
|
||||
past := time.Now().UTC().Add(-time.Hour)
|
||||
post := models.Post{
|
||||
Title: "Published searchable",
|
||||
Slug: "published-searchable",
|
||||
Content: "hello search gate",
|
||||
Published: true,
|
||||
PublishedAt: &past,
|
||||
}
|
||||
if err := env.gdb.Create(&post).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if hits.Load() != 0 {
|
||||
t.Fatalf("Typesense HTTP hits = %d, want 0", hits.Load())
|
||||
}
|
||||
if post.ShouldBeSearchable() {
|
||||
t.Fatal("ShouldBeSearchable true with default search_use_typesense")
|
||||
}
|
||||
draft := models.Post{Title: "Draft", Slug: "draft-search", Content: "x", Published: false}
|
||||
if draft.ShouldBeSearchable() {
|
||||
t.Fatal("unpublished ShouldBeSearchable must be false")
|
||||
}
|
||||
}
|
||||
|
||||
func TestJournalRSS(t *testing.T) {
|
||||
env := newPostsEnv(t)
|
||||
title := "Acme Journal"
|
||||
if err := env.gdb.Model(&models.Settings{}).Where("id = 1").Updates(map[string]any{
|
||||
"rss_title": title,
|
||||
"rss_posts_per_feed": 2,
|
||||
"rss_include_content": false,
|
||||
"rss_enabled": true,
|
||||
}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
past := time.Now().UTC().Add(-time.Hour)
|
||||
for i, slug := range []string{"one", "two", "three"} {
|
||||
p := models.Post{
|
||||
Title: "Post " + slug,
|
||||
Slug: slug,
|
||||
Content: "body " + slug,
|
||||
ContentHTML: "<p>body " + slug + "</p>",
|
||||
Published: true,
|
||||
PublishedAt: ptrTime(past.Add(-time.Duration(i) * time.Minute)),
|
||||
}
|
||||
if err := env.gdb.Create(&p).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := env.gdb.Create(&models.Post{Title: "Draft", Slug: "draft-rss", Content: "no", Published: false}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rec := env.call(t, http.MethodGet, "/_journal/api/v1/rss", "", nil)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("GET rss = %d, want 200: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
ct := rec.Header().Get("Content-Type")
|
||||
if !strings.Contains(ct, "application/rss+xml") {
|
||||
t.Fatalf("Content-Type = %q, want application/rss+xml", ct)
|
||||
}
|
||||
type rssXML struct {
|
||||
XMLName xml.Name `xml:"rss"`
|
||||
Version string `xml:"version,attr"`
|
||||
Channel struct {
|
||||
Title string `xml:"title"`
|
||||
Items []struct {
|
||||
Title string `xml:"title"`
|
||||
} `xml:"item"`
|
||||
} `xml:"channel"`
|
||||
}
|
||||
var doc rssXML
|
||||
if err := xml.Unmarshal(rec.Body.Bytes(), &doc); err != nil {
|
||||
t.Fatalf("rss xml: %v body=%s", err, rec.Body.String())
|
||||
}
|
||||
if doc.Version != "2.0" {
|
||||
t.Fatalf("rss version = %q, want 2.0", doc.Version)
|
||||
}
|
||||
if doc.Channel.Title != title {
|
||||
t.Fatalf("channel title = %q, want %q", doc.Channel.Title, title)
|
||||
}
|
||||
if len(doc.Channel.Items) != 2 {
|
||||
t.Fatalf("item count = %d, want 2 (rss_posts_per_feed): %s", len(doc.Channel.Items), rec.Body.String())
|
||||
}
|
||||
for _, item := range doc.Channel.Items {
|
||||
if item.Title == "Draft" {
|
||||
t.Fatal("unpublished post appeared in RSS")
|
||||
}
|
||||
}
|
||||
if strings.Contains(rec.Body.String(), "content:encoded") {
|
||||
t.Fatal("rss_include_content false still emitted content:encoded")
|
||||
}
|
||||
|
||||
if err := env.gdb.Model(&models.Settings{}).Where("id = 1").Update("rss_enabled", false).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
disabled := env.call(t, http.MethodGet, "/_journal/api/v1/rss", "", nil)
|
||||
if disabled.Code != http.StatusOK {
|
||||
t.Fatalf("rss_enabled false = %d, want 200", disabled.Code)
|
||||
}
|
||||
var disabledDoc rssXML
|
||||
if err := xml.Unmarshal(disabled.Body.Bytes(), &disabledDoc); err != nil {
|
||||
t.Fatalf("rss_enabled false is not well-formed: %v %s", err, disabled.Body.String())
|
||||
}
|
||||
if disabledDoc.Version != "2.0" || disabledDoc.Channel.Title == "" {
|
||||
t.Fatalf("rss_enabled false missing channel: %+v", disabledDoc)
|
||||
}
|
||||
}
|
||||
|
||||
func ptrTime(t time.Time) *time.Time { return &t }
|
||||
80
models/post_search.go
Normal file
80
models/post_search.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/beachcomber"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
var _ beachcomber.Searchable = (*Post)(nil)
|
||||
|
||||
// SearchableAs is the Typesense collection name (D-12).
|
||||
func (Post) SearchableAs() string { return "golem15_journal_posts" }
|
||||
|
||||
// ShouldBeSearchable is false when the post is unpublished or the
|
||||
// beachcomber Gate (search_use_typesense) is off.
|
||||
func (p *Post) ShouldBeSearchable() bool {
|
||||
if p == nil || !p.Published || p.PublishedAt == nil || !p.PublishedAt.Before(time.Now()) {
|
||||
return false
|
||||
}
|
||||
return SearchGateEnabled()
|
||||
}
|
||||
|
||||
// ToSearchableArray builds the Typesense document after commit.
|
||||
func (p *Post) ToSearchableArray(ctx context.Context, db *gorm.DB) (map[string]any, error) {
|
||||
if p == nil {
|
||||
return nil, nil
|
||||
}
|
||||
published := 0
|
||||
if p.Published && p.PublishedAt != nil && p.PublishedAt.Before(time.Now()) {
|
||||
published = 1
|
||||
}
|
||||
pinned := 0
|
||||
if p.IsPinned {
|
||||
pinned = 1
|
||||
}
|
||||
var publishedAt int64
|
||||
if p.PublishedAt != nil {
|
||||
publishedAt = p.PublishedAt.Unix()
|
||||
}
|
||||
var createdAt int64
|
||||
if p.CreatedAt != nil {
|
||||
createdAt = p.CreatedAt.Unix()
|
||||
}
|
||||
userID := 0
|
||||
if p.UserID != nil {
|
||||
userID = int(*p.UserID)
|
||||
}
|
||||
catIDs := make([]int, 0, len(p.Categories))
|
||||
tagIDs := make([]int, 0, len(p.Tags))
|
||||
if db != nil {
|
||||
var cats []PostCategory
|
||||
if err := db.WithContext(ctx).Where("post_id = ?", p.ID).Find(&cats).Error; err == nil {
|
||||
for _, c := range cats {
|
||||
catIDs = append(catIDs, int(c.CategoryID))
|
||||
}
|
||||
}
|
||||
var tags []PostTag
|
||||
if err := db.WithContext(ctx).Where("post_id = ?", p.ID).Find(&tags).Error; err == nil {
|
||||
for _, t := range tags {
|
||||
tagIDs = append(tagIDs, int(t.TagID))
|
||||
}
|
||||
}
|
||||
}
|
||||
return map[string]any{
|
||||
"id": strconv.FormatUint(uint64(p.ID), 10),
|
||||
"published": published,
|
||||
"published_at": publishedAt,
|
||||
"is_pinned": pinned,
|
||||
"user_id": userID,
|
||||
"created_at": createdAt,
|
||||
"category_ids": catIDs,
|
||||
"tag_ids": tagIDs,
|
||||
"title": p.Title,
|
||||
"excerpt": p.Excerpt,
|
||||
"content": p.Content,
|
||||
}, nil
|
||||
}
|
||||
12
models/search_gate.go
Normal file
12
models/search_gate.go
Normal file
@@ -0,0 +1,12 @@
|
||||
package models
|
||||
|
||||
import "sync/atomic"
|
||||
|
||||
var searchGate atomic.Bool
|
||||
|
||||
// SetSearchGate records whether beachcomber.Gate last read search_use_typesense
|
||||
// as on. Read errors and a missing row count as off.
|
||||
func SetSearchGate(on bool) { searchGate.Store(on) }
|
||||
|
||||
// SearchGateEnabled reports the last installed search gate value.
|
||||
func SearchGateEnabled() bool { return searchGate.Load() }
|
||||
@@ -48,7 +48,7 @@ func (p *Plugin) Register(app *backpack.App) error {
|
||||
|
||||
func (p *Plugin) Boot(app *backpack.App) error {
|
||||
p.app = app
|
||||
return nil
|
||||
return p.wireSearch(app)
|
||||
}
|
||||
|
||||
func (p *Plugin) LangFS() fs.FS { return langFS }
|
||||
|
||||
@@ -151,7 +151,7 @@ func adminAPI(rel string) string {
|
||||
return cabana.DefaultAdminPrefix + "/api/v1" + rel
|
||||
}
|
||||
|
||||
func newPostsEnv(t *testing.T) *adminEnv {
|
||||
func newPostsEnv(t *testing.T, extraEnv ...string) *adminEnv {
|
||||
t.Helper()
|
||||
sqlDB := dedicatedDB(t, "journal_admin_"+strconv.FormatInt(time.Now().UnixNano(), 10))
|
||||
gdb, err := lagoon.Use(t.Context(), sqlDB)
|
||||
@@ -164,10 +164,10 @@ func newPostsEnv(t *testing.T) *adminEnv {
|
||||
}
|
||||
cfg, err := compass.Open(compass.Options{
|
||||
Dir: dir,
|
||||
Environ: []string{
|
||||
Environ: append([]string{
|
||||
"SUMMER_ENV=development",
|
||||
"SUMMER_ADMIN__JWT__SECRET=" + adminJWTSecret,
|
||||
},
|
||||
}, extraEnv...),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
|
||||
@@ -19,6 +19,7 @@ func (p *Plugin) Routes(r pact.Router) error {
|
||||
g.Get("/rss", apicontrollers.RSS(p.app))
|
||||
})
|
||||
r.Group("/_journal/api/v1", surf.Use("throttle:journal-api"), func(g pact.Router) {
|
||||
g.Post("/media/upload", apicontrollers.UploadMedia(p.app))
|
||||
g.Post("/posts", apicontrollers.Store(p.app))
|
||||
g.Put("/posts/{id}", apicontrollers.Update(p.app))
|
||||
g.Delete("/posts/{id}", apicontrollers.Destroy(p.app))
|
||||
|
||||
38
search.go
Normal file
38
search.go
Normal file
@@ -0,0 +1,38 @@
|
||||
package journal
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
|
||||
"git.golem15.com/golem15/sm-journal-plugin/models"
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/beachcomber"
|
||||
_ "git.golem15.com/golem15/summercms/modules/beachcomber/typesense"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func (p *Plugin) wireSearch(app *backpack.App) error {
|
||||
svc, err := beachcomber.From(app)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
svc.SetGate(beachcomber.GateFunc(func(ctx context.Context, db *gorm.DB) bool {
|
||||
on := settingsSearchEnabled(ctx, db)
|
||||
models.SetSearchGate(on)
|
||||
return on
|
||||
}))
|
||||
return nil
|
||||
}
|
||||
|
||||
func settingsSearchEnabled(ctx context.Context, db *gorm.DB) bool {
|
||||
if db == nil {
|
||||
return false
|
||||
}
|
||||
var on bool
|
||||
err := db.WithContext(ctx).Raw(`SELECT search_use_typesense FROM golem15_journal_settings WHERE id = 1`).Scan(&on).Error
|
||||
if err != nil {
|
||||
slog.Default().Warn("journal: search_use_typesense read failed; treating as off", "err", err)
|
||||
return false
|
||||
}
|
||||
return on
|
||||
}
|
||||
Reference in New Issue
Block a user