feat(postcard): database overrides for mail templates with file fallback

- postcard owns the system_mail_* tables (postcard.Migrations), run by lagoon
  as the framework set summercms.postcard after system_files
- Send asks an OverrideSource once per message; NewDBSource reads the
  customized template row through the *gorm.DB published on the app
- a failed lookup, missing tables or an override that does not render or
  fails the safety checks sends the file version and is logged
- the caller's Message.Subject still wins; Mailer, Message, NewMailer,
  Catalog.Register and BootPlugin keep their signatures

Plan 35-01, task 2 (tracer).
This commit is contained in:
Jakub Zych
2026-11-13 10:26:10 +01:00
parent 3198bbe1aa
commit 7e5d2c74af
10 changed files with 932 additions and 30 deletions

View File

@@ -8,11 +8,11 @@ Transactional mail from plugin-owned Markdown templates and layouts, delivered t
## Overview
postcard is the mail layer of SummerCMS. Plugins that implement `pact.HasMailTemplates` ship WinterCMS-shaped mail files under `views/mail/`; at boot, `postcard.Activate` publishes one app-scoped `postcard.Mailer` built from the compass `mail.*` config, and `postcard.BootPlugin` registers each plugin's templates and layouts as the plugin boots. Callers then send a template by its dotted name with a map of variables. It is the counterpart of WinterCMS's `Mail::send` with `views/mail/*.htm` templates and mail layouts.
postcard is the mail layer of SummerCMS. Plugins that implement `pact.HasMailTemplates` ship WinterCMS-shaped mail files under `views/mail/`; at boot, `postcard.Activate` publishes one app-scoped `postcard.Mailer` built from the compass `mail.*` config, and `postcard.BootPlugin` registers each plugin's templates and layouts as the plugin boots. Callers then send a template by its dotted name with a map of variables. postcard also owns the mail tables that hold administrators' edits: when a template has a customized row, the mailer sends that override, and the plugin's file otherwise. It is the counterpart of WinterCMS's `Mail::send` with `views/mail/*.htm` templates and mail layouts.
## Role in the framework
A service module on `backpack`, `compass` and `pact`; `party` registers the templates plugins contribute. It is separate so mail rendering and delivery can be tested and swapped (memory, log, SMTP) without touching the code that sends.
A service module on `backpack`, `compass` and `pact`; `party` registers the templates plugins contribute, and `lagoon` runs `postcard.Migrations` with the framework's other migrations. postcard does not import `lagoon`: it finds the database through `backpack`. It is separate so mail rendering and delivery can be tested and swapped (memory, log, SMTP) without touching the code that sends.
Use cases:
@@ -28,6 +28,7 @@ Use cases:
- Every message gets an HTML part (Markdown converted with goldmark) and a plain-text part. `mail.css` and `mail.brandCss` are inlined into the layout's style block.
- Safety checks: rendered HTML with script, iframe, object or embed tags, inline event handlers or `javascript:`/`vbscript:`/`data:` URLs is rejected; CR/LF in the subject or address headers is rejected; addresses are parsed with `net/mail`.
- Drivers behind the `postcard.Driver` interface: `postcard.MemoryDriver` (keeps messages for tests), `postcard.LogDriver` (logs headers and the text part through `log/slog`, never the HTML or credentials), `postcard.SMTPDriver` (go-mail with an explicit TLS policy) and `postcard.FailDriver` (a deterministic failure for tests).
- Database overrides: a customized row of `system_mail_templates` replaces a template's subject, body and layout. The mailer reads it on every send, with no cache, in every process that publishes a `*gorm.DB`. A missing database, tables that are not migrated yet, or an override that does not render or fails the safety checks sends the file version.
- No locale selection: callers pass the full template name, including any locale suffix.
## Usage
@@ -98,16 +99,64 @@ func TestWelcomeMail(t *testing.T) {
}
```
### Overrides from the admin
postcard owns the tables `system_mail_templates`, `system_mail_layouts`, `system_mail_partials`, `system_mail_brand_settings` and `system_mail_image_library` (`postcard.Migrations`). A row of `system_mail_templates` with `is_custom` set is an override of the template named in its `code`: its `subject` and `body` replace the file's, and the row its `layout_id` points at names the layout. The body has the same format as in the file, Markdown with Go template variables.
`postcard.Activate` gives the mailer the source `postcard.NewDBSource` returns. On every `Send` the mailer asks it once for the template being sent; nothing is cached, so an edit applies to the next message in every process that shares the database, including a worker that never loads the admin.
The subject is taken from the first of these that is set:
1. `postcard.Message.Subject`, the caller's subject;
2. the override's subject;
3. the file's subject.
An override never stops mail:
- With no `*gorm.DB` published, the file version is sent.
- When the lookup fails (no connection, or the tables are not migrated yet), the file version is sent and a warning is logged, at most once a minute per distinct error.
- When the override does not parse, fails while rendering (for example it uses a field of a variable the caller does not pass) or fails the safety checks, the error is logged as `postcard: override failed, sending the file version` with the template name, and the file version is sent.
- When the override's layout no longer exists, the message is rendered in the default layout and a warning is logged.
`postcard.NewMailer` takes a source through `postcard.Options.Overrides`; any `postcard.OverrideSource` works, which is how a test supplies overrides without a database:
```go
package blog
import (
"context"
"git.golem15.com/golem15/summercms/modules/postcard"
)
// fixedOverrides overrides one template with a fixed subject and body.
type fixedOverrides struct{}
func (fixedOverrides) MailOverrides(_ context.Context, template string) (postcard.Overrides, error) {
if template != "acme.blog::mail.welcome" {
return postcard.Overrides{}, nil
}
return postcard.Overrides{Template: &postcard.TemplateOverride{
Subject: "Hello, {{ .name }}",
Body: "Good to see you, **{{ .name }}**.",
}}, nil
}
func newTestMailer(cat *postcard.Catalog, driver postcard.Driver) postcard.Mailer {
return postcard.NewMailer(cat, driver, postcard.Options{Overrides: fixedOverrides{}})
}
```
## API reference
| Identifier | Description |
|------------|-------------|
| `postcard.Activate` | Builds the driver from config and publishes the app-scoped `postcard.Mailer` before plugins boot. |
| `postcard.Activate` | Builds the driver from config and publishes the app-scoped `postcard.Mailer` before plugins boot. The mailer reads overrides through `postcard.NewDBSource`. |
| `postcard.BootPlugin` | Registers a booting plugin's declared templates and layouts with the published mailer. |
| `postcard.Mailer` | Sends a registered template through the configured driver. |
| `postcard.NewMailer` | Builds a mailer from a catalog, a driver and `postcard.Options`. |
| `postcard.Message` | What callers send: template name, recipients, reply-to, variables and an optional subject override. |
| `postcard.Options` | Mailer settings: sender address, CSS and brand CSS. |
| `postcard.Options` | Mailer settings: sender address, CSS, brand CSS and, in `postcard.Options.Overrides`, the source of database overrides (nil sends the file versions). |
| `postcard.Catalog` | Registered templates and layouts, including the built-in default layout. |
| `postcard.NewCatalog` | Returns a catalog holding only the default layout. |
| `postcard.Catalog.Register` | Loads a plugin's templates and layout aliases from an `fs.FS`. |
@@ -118,6 +167,14 @@ func TestWelcomeMail(t *testing.T) {
| `postcard.NewSMTPDriver` | SMTP driver built from a `postcard.SMTPConfig`. |
| `postcard.SMTPConfig` | Host, port, credentials, TLS policy and timeout for the SMTP driver. |
| `postcard.FailDriver` | Driver whose `postcard.FailDriver.Send` always returns `postcard.FailDriver.Err`. |
| `postcard.OverrideSource` | Supplies the database overrides of one render. `postcard.OverrideSource.MailOverrides` takes the template name; an empty name asks for everything except a template override. |
| `postcard.Overrides` | What a source returns: the template override in `postcard.Overrides.Template` (nil when there is none), layout content by code in `postcard.Overrides.Layouts` and partial content by name in `postcard.Overrides.Partials`. |
| `postcard.TemplateOverride` | A template's customized subject, body and layout code. An empty `postcard.TemplateOverride.Layout` keeps the file's layout. |
| `postcard.LayoutOverride` | A layout's name, HTML wrapper, text wrapper and CSS as stored in the database. |
| `postcard.PartialOverride` | A partial's HTML and text sections as stored in the database. |
| `postcard.NewDBSource` | The source over postcard's tables. It looks the `*gorm.DB` up on the app at every call and returns no overrides when none is published. |
| `postcard.Migrations` | The gormigrate set that creates the mail tables; lagoon runs it under the `summercms.postcard` history. |
| `postcard.TemplatesTable`, `postcard.LayoutsTable`, `postcard.PartialsTable`, `postcard.BrandTable`, `postcard.ImageLibraryTable` | The names of the mail tables, for code that defines models over them. |
## Configuration
@@ -151,9 +208,9 @@ mail:
## Dependencies
- SummerCMS modules: [backpack](../backpack/README.md), [compass](../compass/README.md), [diary](../diary/README.md) (component logger `postcard` when the app publishes no logger), [pact](../pact/README.md).
- Third-party: `github.com/wneessen/go-mail` (SMTP), `github.com/yuin/goldmark` (Markdown).
- Third-party: `github.com/wneessen/go-mail` (SMTP), `github.com/yuin/goldmark` (Markdown), `gorm.io/gorm` (reading overrides), `github.com/go-gormigrate/gormigrate/v2` (the mail tables' migrations).
- Standard library: `bytes`, `context`, `embed`, `fmt`, `html/template`, `io/fs`, `log/slog`, `net/mail`, `regexp`, `strings`, `sync`, `text/template`, `time`.
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Mailpit container).
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Mailpit and Postgres containers), `gorm.io/driver/postgres` and `github.com/jackc/pgx/v5`.
## Testing
@@ -161,4 +218,4 @@ mail:
go test ./modules/postcard/...
```
The SMTP integration test starts a Mailpit container through testcontainers-go and needs Docker. Run `go test -short ./modules/postcard/...` to skip it; the remaining tests use the memory and fail drivers and need no external services.
The SMTP integration test starts a Mailpit container and the override smoke test a Postgres container, both through testcontainers-go, so they need Docker. Run `go test -short ./modules/postcard/...` to skip them; the remaining tests use the memory and fail drivers and need no external services.

View File

@@ -5,6 +5,7 @@ import (
"fmt"
"log/slog"
"strings"
"sync"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
@@ -38,24 +39,36 @@ type Options struct {
From string
CSS string
BrandCSS string
// Overrides, when set, is asked once per Send for the database
// overrides of the template being sent. Nil sends the file versions.
Overrides OverrideSource
}
type mailer struct {
catalog *Catalog
driver Driver
opts Options
// source supplies database overrides; nil means files only.
source OverrideSource
// log receives override warnings and failures; nil means moduleLog.
log *slog.Logger
warned warnGate
}
// NewMailer returns a mailer that renders from cat and delivers through driver.
// NewMailer returns a mailer that renders from cat and delivers through
// driver. opts.Overrides, when set, supplies database overrides per send.
func NewMailer(cat *Catalog, driver Driver, opts Options) Mailer {
if cat == nil {
cat = NewCatalog()
}
return &mailer{catalog: cat, driver: driver, opts: opts}
return &mailer{catalog: cat, driver: driver, opts: opts, source: opts.Overrides}
}
// Send renders msg.Template and delivers it. It performs no locale selection;
// the caller must pass the full dotted name, including any -en suffix.
// the caller must pass the full dotted name, including any -en suffix. A
// database override of the template is applied when the mailer has an
// OverrideSource; an override that cannot be read or does not render is
// logged and the file version is sent instead.
func (m *mailer) Send(ctx context.Context, msg Message) error {
if m == nil {
return fmt.Errorf("postcard: mailer is nil")
@@ -69,10 +82,7 @@ func (m *mailer) Send(ctx context.Context, msg Message) error {
if len(msg.To) == 0 && len(msg.Cc) == 0 && len(msg.Bcc) == 0 {
return fmt.Errorf("postcard: no recipients")
}
rendered, err := m.catalog.render(msg, renderOptions{
css: m.opts.CSS,
brandCSS: m.opts.BrandCSS,
})
rendered, err := m.render(ctx, msg)
if err != nil {
return err
}
@@ -87,8 +97,101 @@ func (m *mailer) Send(ctx context.Context, msg Message) error {
return nil
}
// render renders msg with its database overrides applied. With no source, no
// overrides, or overrides that fail to render or to pass the safety checks,
// it renders the file version.
func (m *mailer) render(ctx context.Context, msg Message) (RenderedMessage, error) {
file := renderOptions{css: m.opts.CSS, brandCSS: m.opts.BrandCSS}
if _, err := m.catalog.lookupTemplate(msg.Template); err != nil {
return RenderedMessage{}, err
}
ov := m.overrides(ctx, msg.Template)
if ov.empty() {
return m.catalog.render(msg, file)
}
with := file
with.overrides = ov
with.warn = m.warn
rendered, err := m.catalog.render(msg, with)
if err == nil {
return rendered, nil
}
// The override is at fault only when the file version renders; a
// failure of both is the caller's (a bad subject, a missing variable)
// and is returned as before.
rendered, fileErr := m.catalog.render(msg, file)
if fileErr != nil {
return RenderedMessage{}, fileErr
}
m.logger().Error("postcard: override failed, sending the file version",
slog.String("template", msg.Template),
slog.String("error", err.Error()),
)
return rendered, nil
}
// overrides asks the source once. A failed lookup (no connection, tables not
// migrated yet) is a warning, not a send failure: the file version is sent.
func (m *mailer) overrides(ctx context.Context, template string) Overrides {
if m.source == nil {
return Overrides{}
}
ov, err := m.source.MailOverrides(ctx, template)
if err != nil {
m.warn(err.Error(), "postcard: mail overrides unavailable, sending the file version",
slog.String("template", template),
slog.String("error", err.Error()),
)
return Overrides{}
}
return ov
}
func (m *mailer) logger() *slog.Logger {
if m.log != nil {
return m.log
}
return moduleLog
}
// warn logs at warning level, at most once per warnInterval per key.
func (m *mailer) warn(key, msg string, args ...any) {
if !m.warned.allow(key, time.Now()) {
return
}
m.logger().Warn(msg, args...)
}
const (
// warnInterval is how often the same override warning is repeated.
warnInterval = time.Minute
// warnGateSize bounds the keys a warnGate remembers.
warnGateSize = 64
)
// warnGate rate-limits warnings by key. The zero value is ready to use.
type warnGate struct {
mu sync.Mutex
last map[string]time.Time
}
func (g *warnGate) allow(key string, now time.Time) bool {
g.mu.Lock()
defer g.mu.Unlock()
if last, ok := g.last[key]; ok && now.Sub(last) < warnInterval {
return false
}
if g.last == nil || len(g.last) >= warnGateSize {
g.last = make(map[string]time.Time)
}
g.last[key] = now
return true
}
// Activate publishes an app-scoped Mailer before plugin Boot. Catalog files
// are validated later at each plugin's Boot transition.
// are validated later at each plugin's Boot transition. The mailer reads
// database overrides through NewDBSource, so they apply in every process
// that publishes a *gorm.DB on app.
func Activate[P interface{ ID() string }](app *backpack.App, _ []P) error {
if app == nil {
return fmt.Errorf("postcard: app is nil")
@@ -97,10 +200,12 @@ func Activate[P interface{ ID() string }](app *backpack.App, _ []P) error {
if err != nil {
return err
}
opts.Overrides = NewDBSource(app)
m := &mailer{
catalog: NewCatalog(),
driver: driver,
opts: opts,
source: opts.Overrides,
}
if err := app.Publish[Mailer](m); err != nil {
return fmt.Errorf("postcard: %w", err)

View File

@@ -0,0 +1,450 @@
package postcard
import (
"bytes"
"context"
"database/sql"
"fmt"
"io/fs"
"log/slog"
"net/url"
"strings"
"sync"
"sync/atomic"
"testing"
"testing/fstest"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"github.com/go-gormigrate/gormigrate/v2"
_ "github.com/jackc/pgx/v5/stdlib"
tcpostgres "github.com/testcontainers/testcontainers-go/modules/postgres"
gormpostgres "gorm.io/driver/postgres"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
var (
mailPGOnce sync.Once
mailPGAdmin *sql.DB
mailPGDSN string
mailPGErr error
mailPGSeq atomic.Int64
)
// mailPostgres returns a GORM handle on a fresh database with
// postcard.Migrations applied. Every call gets its own database on one
// shared postgres:16-alpine container, so tests cannot see each other's
// rows. It skips under -short. postcard must not import lagoon, so this
// opens Postgres directly.
func mailPostgres(t *testing.T) *gorm.DB {
t.Helper()
if testing.Short() {
t.Skip("requires testcontainers postgres")
}
mailPGOnce.Do(func() {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
ctr, err := tcpostgres.Run(ctx, "postgres:16-alpine",
tcpostgres.WithDatabase("postcard"),
tcpostgres.WithUsername("postcard"),
tcpostgres.WithPassword("postcard"),
tcpostgres.BasicWaitStrategies(),
)
if err != nil {
mailPGErr = err
return
}
dsn, err := ctr.ConnectionString(ctx, "sslmode=disable")
if err != nil {
mailPGErr = err
return
}
db, err := sql.Open("pgx", dsn)
if err != nil {
mailPGErr = err
return
}
if err := db.PingContext(ctx); err != nil {
mailPGErr = err
return
}
mailPGAdmin, mailPGDSN = db, dsn
})
if mailPGErr != nil {
t.Fatalf("postgres: %v", mailPGErr)
}
name := fmt.Sprintf("mail_%d", mailPGSeq.Add(1))
if _, err := mailPGAdmin.Exec("CREATE DATABASE " + name); err != nil {
t.Fatalf("create database: %v", err)
}
u, err := url.Parse(mailPGDSN)
if err != nil {
t.Fatalf("postgres dsn: %v", err)
}
u.Path = "/" + name
sqlDB, err := sql.Open("pgx", u.String())
if err != nil {
t.Fatalf("open %s: %v", name, err)
}
t.Cleanup(func() { _ = sqlDB.Close() })
gdb, err := gorm.Open(gormpostgres.New(gormpostgres.Config{Conn: sqlDB}), &gorm.Config{Logger: logger.Discard})
if err != nil {
t.Fatalf("gorm open: %v", err)
}
if err := gormigrate.New(gdb, gormigrate.DefaultOptions, Migrations).Migrate(); err != nil {
t.Fatalf("migrate: %v", err)
}
return gdb
}
// acmeBlog is a fixture plugin that ships one template and one layout.
type acmeBlog struct{}
func (acmeBlog) ID() string { return "acme.blog" }
func (acmeBlog) MailTemplatesFS() fs.FS {
return fstest.MapFS{
"views/mail/welcome.htm": {Data: []byte(`subject = "Welcome, {{ .name }}"
description = "Sent after registration"
layout = "blog"
==
Hi **{{ .name }}**, thanks for joining.
`)},
"views/mail/layouts/blog.htm": {Data: []byte(`name = "Blog"
==
{{ .Content }}
-- The Acme blog
==
<div class="blog-mail">{{ .Content }}</div>
`)},
}
}
func (acmeBlog) MailTemplates() []string { return []string{"acme.blog::mail.welcome"} }
func (acmeBlog) MailLayouts() map[string]string {
return map[string]string{"blog": "acme.blog::mail.layouts.blog"}
}
const acmeWelcome = "acme.blog::mail.welcome"
// bootMailApp activates postcard on a fresh app the way party does, boots
// the fixture plugin and returns the published mailer with its log captured.
// gdb may be nil: the app then has no database.
func bootMailApp(t *testing.T, gdb *gorm.DB) (*mailer, *MemoryDriver, *bytes.Buffer) {
t.Helper()
app := backpack.New(nil)
if gdb != nil {
if err := app.Publish(gdb); err != nil {
t.Fatal(err)
}
}
plugin := acmeBlog{}
if err := Activate(app, []acmeBlog{plugin}); err != nil {
t.Fatalf("Activate: %v", err)
}
if err := BootPlugin(app, plugin); err != nil {
t.Fatalf("BootPlugin: %v", err)
}
got, ok := app.Lookup[Mailer]()
if !ok {
t.Fatal("mailer not published")
}
m, ok := got.(*mailer)
if !ok {
t.Fatalf("published mailer is %T", got)
}
drv, ok := m.driver.(*MemoryDriver)
if !ok {
t.Fatalf("driver is %T, want the memory driver", m.driver)
}
var logs bytes.Buffer
m.log = slog.New(slog.NewTextHandler(&logs, &slog.HandlerOptions{Level: slog.LevelDebug}))
return m, drv, &logs
}
// customizeWelcome turns the welcome template's index row into an override.
// layoutCode names the layout row to point at; empty leaves layout_id NULL.
func customizeWelcome(t *testing.T, gdb *gorm.DB, subject, body, layoutCode string) {
t.Helper()
var layoutID any
if layoutCode != "" {
var id int64
if err := gdb.Raw(`INSERT INTO `+LayoutsTable+` (code, source, name) VALUES (?, 'acme.blog', 'Blog')
ON CONFLICT (code) DO UPDATE SET name = EXCLUDED.name RETURNING id`, layoutCode).Scan(&id).Error; err != nil {
t.Fatalf("layout row: %v", err)
}
layoutID = id
}
err := gdb.Exec(`INSERT INTO `+TemplatesTable+` (code, source, subject, body, layout_id, is_custom, customized_at)
VALUES (?, 'acme.blog', ?, ?, ?, TRUE, NOW())
ON CONFLICT (code) DO UPDATE SET subject = EXCLUDED.subject, body = EXCLUDED.body,
layout_id = EXCLUDED.layout_id, is_custom = TRUE`, acmeWelcome, subject, body, layoutID).Error
if err != nil {
t.Fatalf("template row: %v", err)
}
}
func sendWelcome(t *testing.T, m Mailer, subject string) {
t.Helper()
err := m.Send(context.Background(), Message{
Template: acmeWelcome,
To: []string{"ada@example.com"},
Vars: map[string]any{"name": "Ada"},
Subject: subject,
})
if err != nil {
t.Fatalf("Send: %v", err)
}
}
func lastMessage(t *testing.T, drv *MemoryDriver) RenderedMessage {
t.Helper()
msgs := drv.Messages()
if len(msgs) == 0 {
t.Fatal("no message was sent")
}
return msgs[len(msgs)-1]
}
// fileWelcome is what a mailer with no override source sends for the
// fixture: the reference every fallback is compared with.
func fileWelcome(t *testing.T) RenderedMessage {
t.Helper()
plugin := acmeBlog{}
cat := NewCatalog()
if err := cat.Register(plugin.ID(), plugin.MailTemplatesFS(), plugin.MailTemplates(), plugin.MailLayouts()); err != nil {
t.Fatal(err)
}
drv := NewMemoryDriver()
sendWelcome(t, NewMailer(cat, drv, Options{}), "")
return lastMessage(t, drv)
}
// TestMailOverrideSmoke is the tracer of the database override layer: a
// process that only ran postcard.Activate (no admin module) and published a
// *gorm.DB sends a customized row instead of the plugin's file, and anything
// wrong with that row or with the database degrades to the file version.
func TestMailOverrideSmoke(t *testing.T) {
if testing.Short() {
t.Skip("requires testcontainers postgres")
}
file := fileWelcome(t)
if file.Subject != "Welcome, Ada" || !strings.Contains(file.HTML, `<div class="blog-mail"><p>Hi <strong>Ada</strong>`) {
t.Fatalf("file version = %+v", file)
}
t.Run("override", func(t *testing.T) {
gdb := mailPostgres(t)
m, drv, logs := bootMailApp(t, gdb)
customizeWelcome(t, gdb, "Hello {{ .name }}", "Good to see you, *{{ .name }}*.", "acme.blog::mail.layouts.blog")
sendWelcome(t, m, "")
got := lastMessage(t, drv)
if got.Subject != "Hello Ada" {
t.Errorf("subject = %q, want the override's", got.Subject)
}
if !strings.Contains(got.HTML, `<div class="blog-mail"><p>Good to see you, <em>Ada</em>.</p>`) {
t.Errorf("HTML = %q, want the override body in the blog layout", got.HTML)
}
if got.Text != "Good to see you, *Ada*.\n\n-- The Acme blog" {
t.Errorf("text = %q", got.Text)
}
if logs.Len() != 0 {
t.Errorf("unexpected log output:\n%s", logs)
}
// No cache: an edit and a reset are both effective on the next send.
customizeWelcome(t, gdb, "Second {{ .name }}", "Edited.", "acme.blog::mail.layouts.blog")
sendWelcome(t, m, "")
if got := lastMessage(t, drv); got.Subject != "Second Ada" {
t.Errorf("subject after an edit = %q", got.Subject)
}
if err := gdb.Exec(`UPDATE `+TemplatesTable+` SET is_custom = FALSE WHERE code = ?`, acmeWelcome).Error; err != nil {
t.Fatal(err)
}
sendWelcome(t, m, "")
if got := lastMessage(t, drv); got.Subject != file.Subject || got.HTML != file.HTML || got.Text != file.Text {
t.Errorf("after is_custom was cleared the row still applies: %+v", got)
}
})
t.Run("override_layout", func(t *testing.T) {
gdb := mailPostgres(t)
m, drv, logs := bootMailApp(t, gdb)
// The override moves the template to postcard's default layout.
customizeWelcome(t, gdb, "Hello", "Plain.", defaultLayoutName)
sendWelcome(t, m, "")
got := lastMessage(t, drv)
if !strings.Contains(got.HTML, `<div class="content-body"><p>Plain.</p>`) || strings.Contains(got.HTML, "blog-mail") {
t.Errorf("HTML = %q, want the default layout", got.HTML)
}
if logs.Len() != 0 {
t.Errorf("unexpected log output:\n%s", logs)
}
// A layout code that names nothing renders with the default layout
// and says so.
customizeWelcome(t, gdb, "Hello", "Plain.", "gone")
sendWelcome(t, m, "")
got = lastMessage(t, drv)
if !strings.Contains(got.HTML, `<div class="content-body"><p>Plain.</p>`) {
t.Errorf("HTML = %q, want the default layout for an unknown layout code", got.HTML)
}
if !strings.Contains(logs.String(), "level=WARN") || !strings.Contains(logs.String(), "unknown layout") || !strings.Contains(logs.String(), "layout=gone") {
t.Errorf("log = %q, want a warning naming the layout", logs)
}
})
t.Run("file", func(t *testing.T) {
gdb := mailPostgres(t)
m, drv, logs := bootMailApp(t, gdb)
sendWelcome(t, m, "")
got := lastMessage(t, drv)
if got.Subject != file.Subject || got.HTML != file.HTML || got.Text != file.Text {
t.Errorf("without a row the mail differs from the file version:\n got %+v\nwant %+v", got, file)
}
// A row that is not customized (an index row) changes nothing.
if err := gdb.Exec(`INSERT INTO `+TemplatesTable+` (code, source, subject, body) VALUES (?, 'acme.blog', 'Mirror', 'Mirror')`, acmeWelcome).Error; err != nil {
t.Fatal(err)
}
sendWelcome(t, m, "")
got = lastMessage(t, drv)
if got.Subject != file.Subject || got.HTML != file.HTML || got.Text != file.Text {
t.Errorf("a row with is_custom false changed the mail: %+v", got)
}
if logs.Len() != 0 {
t.Errorf("unexpected log output:\n%s", logs)
}
})
t.Run("fallback", func(t *testing.T) {
gdb := mailPostgres(t)
m, drv, logs := bootMailApp(t, gdb)
cases := map[string]string{
// A field of a variable the caller does not pass fails at execution.
"runtime": "Order {{ .order.code }}",
// A body that does not parse.
"parse": "Hi {{ .name",
// A body that renders but fails the safety checks on the result.
"unsafe": "Type javascript: in the address bar.",
}
for name, body := range cases {
logs.Reset()
customizeWelcome(t, gdb, "Hello {{ .name }}", body, "acme.blog::mail.layouts.blog")
sendWelcome(t, m, "")
got := lastMessage(t, drv)
if got.Subject != file.Subject || got.HTML != file.HTML || got.Text != file.Text {
t.Errorf("%s: a failing override did not fall back to the file version: %+v", name, got)
}
out := logs.String()
if !strings.Contains(out, "level=ERROR") || !strings.Contains(out, "postcard: override failed, sending the file version") ||
!strings.Contains(out, "template="+acmeWelcome) || !strings.Contains(out, "error=") {
t.Errorf("%s: log = %q, want the override failure with template and error", name, out)
}
}
// A subject override with a line break is refused the same way.
logs.Reset()
customizeWelcome(t, gdb, "Hello\nBcc: all@example.com", "Fine.", "acme.blog::mail.layouts.blog")
sendWelcome(t, m, "")
if got := lastMessage(t, drv); got.Subject != file.Subject || got.HTML != file.HTML {
t.Errorf("a subject override with a line break was sent: %+v", got)
}
if !strings.Contains(logs.String(), "CR/LF") {
t.Errorf("log = %q, want the CR/LF refusal", logs)
}
})
t.Run("caller_subject", func(t *testing.T) {
gdb := mailPostgres(t)
m, drv, _ := bootMailApp(t, gdb)
customizeWelcome(t, gdb, "Hello {{ .name }}", "Override body.", "acme.blog::mail.layouts.blog")
sendWelcome(t, m, "From the caller, {{ .name }}")
got := lastMessage(t, drv)
if got.Subject != "From the caller, Ada" {
t.Errorf("subject = %q, want the caller's", got.Subject)
}
if !strings.Contains(got.HTML, "Override body.") {
t.Errorf("HTML = %q, want the override body next to the caller's subject", got.HTML)
}
// A caller subject that is refused fails the send as before, and is
// not blamed on the override.
before := len(drv.Messages())
err := m.Send(context.Background(), Message{
Template: acmeWelcome,
To: []string{"ada@example.com"},
Subject: "Hi\r\nBcc: all@example.com",
})
if err == nil || !strings.Contains(err.Error(), "CR/LF") {
t.Errorf("Send with a CR/LF caller subject = %v", err)
}
if len(drv.Messages()) != before {
t.Error("a refused message was sent")
}
})
t.Run("missing_table", func(t *testing.T) {
gdb := mailPostgres(t)
m, drv, logs := bootMailApp(t, gdb)
customizeWelcome(t, gdb, "Hello", "Override.", "")
if err := gdb.Exec(`DROP TABLE ` + TemplatesTable).Error; err != nil {
t.Fatal(err)
}
for range 3 {
sendWelcome(t, m, "")
got := lastMessage(t, drv)
if got.Subject != file.Subject || got.HTML != file.HTML || got.Text != file.Text {
t.Fatalf("with the table missing the mail differs from the file version: %+v", got)
}
}
out := logs.String()
if n := strings.Count(out, "postcard: mail overrides unavailable, sending the file version"); n != 1 {
t.Errorf("warning logged %d times for 3 sends, want 1 (rate-limited):\n%s", n, out)
}
if !strings.Contains(out, "level=WARN") || !strings.Contains(out, TemplatesTable) {
t.Errorf("log = %q, want a warning naming the table", out)
}
})
t.Run("no_db", func(t *testing.T) {
m, drv, logs := bootMailApp(t, nil)
sendWelcome(t, m, "")
got := lastMessage(t, drv)
if got.Subject != file.Subject || got.HTML != file.HTML || got.Text != file.Text {
t.Errorf("without a database the mail differs from the file version: %+v", got)
}
if logs.Len() != 0 {
t.Errorf("unexpected log output:\n%s", logs)
}
})
t.Run("unknown_template", func(t *testing.T) {
m, _, _ := bootMailApp(t, mailPostgres(t))
err := m.Send(context.Background(), Message{Template: "acme.blog::mail.missing", To: []string{"ada@example.com"}})
if err == nil || !strings.Contains(err.Error(), "unknown template acme.blog::mail.missing") {
t.Errorf("Send of an unknown template = %v", err)
}
})
t.Run("migrations_roll_back", func(t *testing.T) {
gdb := mailPostgres(t)
mig := gormigrate.New(gdb, gormigrate.DefaultOptions, Migrations)
if err := mig.RollbackLast(); err != nil {
t.Fatalf("rollback: %v", err)
}
var n int64
if err := gdb.Raw(`SELECT COUNT(*) FROM information_schema.tables WHERE table_name LIKE 'system_mail_%'`).Scan(&n).Error; err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("%d mail tables left after the rollback", n)
}
if err := mig.Migrate(); err != nil {
t.Fatalf("migrate again: %v", err)
}
})
}

221
modules/postcard/store.go Normal file
View File

@@ -0,0 +1,221 @@
package postcard
import (
"context"
"fmt"
"git.golem15.com/golem15/summercms/modules/backpack"
"github.com/go-gormigrate/gormigrate/v2"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// The mail tables postcard owns. Admin code builds its models over these
// names; postcard itself reads them with plain SQL.
const (
// TemplatesTable holds one row per mail template.
TemplatesTable = "system_mail_templates"
// LayoutsTable holds one row per mail layout, file-backed or database-only.
LayoutsTable = "system_mail_layouts"
// PartialsTable holds one row per mail partial.
PartialsTable = "system_mail_partials"
// BrandTable holds the mail branding settings (a single row, id 1).
BrandTable = "system_mail_brand_settings"
// ImageLibraryTable holds the one row the mail image library attaches
// its files to.
ImageLibraryTable = "system_mail_image_library"
)
// Migrations creates the mail tables. lagoon runs it as a framework set
// under the summercms.postcard history, after the system_files set.
//
// A row is an override only while is_custom is true. file_hash is the hash
// of the current file and base_hash the hash of the file the override was
// written against, so a customized row is outdated when the two differ.
var Migrations = []*gormigrate.Migration{
{
ID: "202611060001_create_system_mail_tables",
Migrate: func(tx *gorm.DB) error {
stmts := []string{
`CREATE TABLE ` + LayoutsTable + ` (
id SERIAL PRIMARY KEY,
code TEXT NOT NULL UNIQUE,
source TEXT,
name TEXT NOT NULL,
content_html TEXT NOT NULL DEFAULT '',
content_text TEXT NOT NULL DEFAULT '',
content_css TEXT NOT NULL DEFAULT '',
is_custom BOOLEAN NOT NULL DEFAULT FALSE,
file_hash TEXT,
base_hash TEXT,
customized_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)`,
`CREATE TABLE ` + TemplatesTable + ` (
id SERIAL PRIMARY KEY,
code TEXT NOT NULL UNIQUE,
source TEXT NOT NULL,
locale TEXT,
description TEXT NOT NULL DEFAULT '',
subject TEXT NOT NULL DEFAULT '',
body TEXT NOT NULL DEFAULT '',
layout_id INTEGER REFERENCES ` + LayoutsTable + `(id) ON DELETE RESTRICT,
is_custom BOOLEAN NOT NULL DEFAULT FALSE,
file_hash TEXT,
base_hash TEXT,
customized_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)`,
`CREATE INDEX ` + TemplatesTable + `_layout_id_index ON ` + TemplatesTable + ` (layout_id)`,
`CREATE TABLE ` + PartialsTable + ` (
id SERIAL PRIMARY KEY,
code TEXT NOT NULL UNIQUE,
source TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
content_html TEXT NOT NULL DEFAULT '',
content_text TEXT NOT NULL DEFAULT '',
is_custom BOOLEAN NOT NULL DEFAULT FALSE,
file_hash TEXT,
base_hash TEXT,
customized_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)`,
`CREATE TABLE ` + BrandTable + ` (
id INTEGER PRIMARY KEY,
logo_file_id INTEGER,
body_bg TEXT NOT NULL DEFAULT '',
content_bg TEXT NOT NULL DEFAULT '',
header_color TEXT NOT NULL DEFAULT '',
footer_color TEXT NOT NULL DEFAULT '',
heading_color TEXT NOT NULL DEFAULT '',
text_color TEXT NOT NULL DEFAULT '',
link_color TEXT NOT NULL DEFAULT '',
button_text_color TEXT NOT NULL DEFAULT '',
button_primary_bg TEXT NOT NULL DEFAULT '',
button_positive_bg TEXT NOT NULL DEFAULT '',
button_negative_bg TEXT NOT NULL DEFAULT '',
panel_bg TEXT NOT NULL DEFAULT '',
table_border_color TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)`,
`CREATE TABLE ` + ImageLibraryTable + ` (
id SERIAL PRIMARY KEY,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)`,
}
for _, stmt := range stmts {
if err := tx.Exec(stmt).Error; err != nil {
return err
}
}
return nil
},
Rollback: func(tx *gorm.DB) error {
// Templates first: they reference layouts.
for _, table := range []string{ImageLibraryTable, BrandTable, PartialsTable, TemplatesTable, LayoutsTable} {
if err := tx.Exec("DROP TABLE IF EXISTS " + table).Error; err != nil {
return err
}
}
return nil
},
},
}
// OverrideSource supplies the database overrides one render needs.
type OverrideSource interface {
// MailOverrides returns the overrides for one send of template. An
// empty template asks for everything except a template override. A nil
// or empty field means the file version is used.
MailOverrides(ctx context.Context, template string) (Overrides, error)
}
// Overrides is what an OverrideSource returns for one render.
type Overrides struct {
// Template is the customized subject, body and layout of the template
// being sent, or nil.
Template *TemplateOverride
// Layouts holds layout content by layout code.
Layouts map[string]LayoutOverride
// Partials holds partial content by partial name.
Partials map[string]PartialOverride
}
// empty reports whether o changes nothing about a render.
func (o Overrides) empty() bool {
return o.Template == nil && len(o.Layouts) == 0 && len(o.Partials) == 0
}
// TemplateOverride replaces a template's subject, body and layout.
type TemplateOverride struct {
Subject string
Body string
// Layout is a layout code: the full name of a file layout or the code
// of a database-only one. Empty keeps the file's layout.
Layout string
}
// LayoutOverride replaces a layout's wrappers and adds its CSS.
type LayoutOverride struct {
Name string
HTML string
Text string
CSS string
}
// PartialOverride replaces a partial's HTML and text sections.
type PartialOverride struct {
HTML string
Text string
}
// NewDBSource returns the OverrideSource that reads postcard's tables. It
// looks the *gorm.DB up on app at every call and never caches a row, so an
// edit is effective on the next send in every process that shares the
// database. With no database published it returns no overrides and no error.
func NewDBSource(app *backpack.App) OverrideSource {
return &dbSource{app: app}
}
type dbSource struct {
app *backpack.App
}
// MailOverrides implements OverrideSource. name is the template's full name.
func (s *dbSource) MailOverrides(ctx context.Context, name string) (Overrides, error) {
var out Overrides
if s == nil || s.app == nil {
return out, nil
}
gdb, ok := s.app.Lookup[*gorm.DB]()
if !ok || gdb == nil {
return out, nil
}
// The mailer reports a failed lookup itself, rate-limited; GORM's own
// error line per send would repeat it for every message.
db := gdb.Session(&gorm.Session{Logger: gdb.Logger.LogMode(logger.Silent)}).WithContext(ctx)
if name == "" {
return out, nil
}
var rows []struct {
Subject string
Body string
Layout string
}
err := db.Raw(`SELECT t.subject, t.body, COALESCE(l.code, '') AS layout
FROM `+TemplatesTable+` t
LEFT JOIN `+LayoutsTable+` l ON l.id = t.layout_id
WHERE t.code = ? AND t.is_custom`, name).Scan(&rows).Error
if err != nil {
return Overrides{}, fmt.Errorf("postcard: mail overrides: %w", err)
}
if len(rows) > 0 {
out.Template = &TemplateOverride{Subject: rows[0].Subject, Body: rows[0].Body, Layout: rows[0].Layout}
}
return out, nil
}

View File

@@ -317,30 +317,63 @@ func parseINI(header string) (map[string]string, error) {
type renderOptions struct {
css string
brandCSS string
// overrides are the database overrides to apply; the zero value renders
// the file version.
overrides Overrides
// warn reports a problem with an override that the render recovered
// from. key identifies the problem for rate limiting. Nil drops it.
warn func(key, msg string, args ...any)
}
// renderSource is what one render works from: a file template, or a file
// template with its override applied.
type renderSource struct {
name string // template name, used in parse and execute errors
subject string
body string
layout string // full layout name, the default alias, or an override's layout code
// layoutFromOverride marks a layout an override chose: one that resolves
// to nothing renders with the default layout instead of failing.
layoutFromOverride bool
}
// render renders msg.Template: the file version, or the file version with
// opts.overrides applied. Both go through renderFrom.
func (c *Catalog) render(msg Message, opts renderOptions) (RenderedMessage, error) {
var out RenderedMessage
t, err := c.lookupTemplate(msg.Template)
if err != nil {
return out, err
return RenderedMessage{}, err
}
src := renderSource{name: t.name, subject: t.subject, body: t.body, layout: t.layout}
if ov := opts.overrides.Template; ov != nil {
src.subject, src.body = ov.Subject, ov.Body
if ov.Layout != "" {
src.layout, src.layoutFromOverride = ov.Layout, true
}
}
return c.renderFrom(msg, src, opts)
}
// renderFrom renders src for msg. The caller's non-empty msg.Subject wins
// over src.subject.
func (c *Catalog) renderFrom(msg Message, src renderSource, opts renderOptions) (RenderedMessage, error) {
var out RenderedMessage
vars := msg.Vars
if vars == nil {
vars = map[string]any{}
}
subjectSrc := t.subject
subjectSrc := src.subject
if strings.TrimSpace(msg.Subject) != "" {
subjectSrc = msg.Subject
}
subject, err := execHTML(t.name+".subject", subjectSrc, vars)
subject, err := execHTML(src.name+".subject", subjectSrc, vars)
if err != nil {
return out, err
}
if strings.ContainsAny(subject, "\r\n") {
return out, fmt.Errorf("postcard: subject contains CR/LF")
}
markdownBody, err := execHTML(t.name+".body", t.body, vars)
markdownBody, err := execHTML(src.name+".body", src.body, vars)
if err != nil {
return out, err
}
@@ -351,7 +384,7 @@ func (c *Catalog) render(msg Message, opts renderOptions) (RenderedMessage, erro
if err := validateHTML(htmlBody); err != nil {
return out, err
}
layout, err := c.lookupLayout(t.layout)
layout, err := c.layoutFor(src, opts)
if err != nil {
return out, err
}
@@ -388,6 +421,22 @@ func (c *Catalog) render(msg Message, opts renderOptions) (RenderedMessage, erro
return out, nil
}
// layoutFor returns the layout src renders in. A layout code from an
// override that names no layout is reported through opts.warn and replaced
// by the default layout.
func (c *Catalog) layoutFor(src renderSource, opts renderOptions) (*parsedLayout, error) {
layout, err := c.lookupLayout(src.layout)
if err == nil || !src.layoutFromOverride {
return layout, err
}
if opts.warn != nil {
opts.warn("layout:"+src.name+":"+src.layout,
"postcard: override names an unknown layout, using the default layout",
"template", src.name, "layout", src.layout)
}
return c.lookupLayout(defaultLayoutAlias)
}
func markdownHTML(src string) (string, error) {
var buf bytes.Buffer
if err := markdown.Convert([]byte(src), &buf); err != nil {