golem15/summercms
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(lagoon): WebP variants of attachment thumbnails

Browse Source 
- attach.VariantKey: a thumbnail's copy in another format next to it
- attach/webp (separate package, so only binaries that import it link
  the encoder): Encode at quality 80 and Variant, which writes the WebP
  copy of a thumbnail once and leaves no file on a failed encode
- github.com/gen2brain/webp v0.6.4, decided in
  .planning/notes/2026-10-28-webp-gen2brain.md
- lagoon README and docs/database/attachments.md
This commit is contained in:
Jakub Zych
2026-11-04 16:52:59 +01:00
parent 1bb2ecf9cc
commit 5c0722429b
8 changed files with 401 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/database/attachments.md
View File

@@ -89,6 +89,44 @@ Originals in JPEG, PNG, GIF and WebP can be thumbnailed. The thumbnailer cannot
> [!WARNING]
> Serve uploads from a separate origin, or at least never mount the ungated handler on the application's own origin. An uploaded file served with its own content type from the API's origin can run script in that origin.
## WebP variants
The `attach/webp` package writes a WebP copy of a thumbnail next to it, so a page can offer WebP with the original format as fallback. `webp.Variant` takes the same width, height and mode as `attach.File.ThumbKey`, makes the thumbnail first when it is missing, encodes it at quality 80 and stores it under `attach.VariantKey`: `thumb_42_480_0_0_0_auto.jpg` gets `thumb_42_480_0_0_0_auto.webp`, with the `image/webp` content type. A thumbnail of a `.webp` original already ends in `.webp`, so its variant is `thumb_42_480_0_0_0_auto.webp.webp`. Variants live under the file's thumbnail prefix, so `attach.DeleteKeys` removes them with the thumbnails.
```go src=modules/lagoon/attach/webp/webp_smoke_test.go#ExampleVariant
ctx := context.Background()
bucket := memblob.OpenBucket(nil)
defer bucket.Close()
// The system_files row of a cover image and its stored original.
f := &attach.File{ID: 42, DiskName: "5f1d0c2e9a7b4c3d8e6f.jpg", FileName: "cover.jpg"}
var img bytes.Buffer
if err := jpeg.Encode(&img, image.NewRGBA(image.Rect(0, 0, 1600, 1200)), nil); err != nil {
fmt.Println(err)
return
}
if err := bucket.WriteAll(ctx, attach.BlobKey(f.DiskName), img.Bytes(), nil); err != nil {
fmt.Println(err)
return
}
// In a background job: the 800 px thumbnail and its WebP copy.
key, err := webp.Variant(ctx, bucket, f, 800, 800, "auto")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(key)
fmt.Println(attach.PublicURL(key))
// Output:
// 5f1/d0c/2e9/thumb_42_800_800_0_0_auto.webp
// /storage/uploads/5f1/d0c/2e9/thumb_42_800_800_0_0_auto.webp
```
A variant that exists is returned without writing, so a job can run again safely. The encode happens in memory before anything is written, and a failed write deletes the key, so a failure never leaves a partial file. Encoding takes tens to hundreds of milliseconds per image, so run it in a background [job](../services/jobs.md), never inside an upload request. The original upload is never re-encoded.
The encoder is `github.com/gen2brain/webp`. It loads the system libwebp (`libwebp.so`, `libwebpdemux.so`) through purego when the library is installed, without cgo, and otherwise falls back to libwebp transpiled to Go, about five times slower. Only binaries that import `attach/webp` link it. Such a binary is dynamically linked against glibc, even with `CGO_ENABLED=0`, so it runs on Debian or Ubuntu hosts but not in a `scratch` or musl image. Build it with `-tags nodynamic` for a static binary that always uses the transpiled encoder.
## Storing an upload
`attach.Store` turns an upload into a `system_files` row. It takes an `attach.Upload` (the client's file name, the body and the public flag) and an `attach.Limits`, writes the bytes under the WinterCMS partition key and inserts an unattached row whose `sort_order` equals its id, as WinterCMS's Sortable trait does. The caller attaches the row later by setting `Field`, `AttachmentType` and `AttachmentID`, usually when the record's form is saved. When the row insert fails, the blob is deleted again.

1
go.mod
View File

@@ -16,6 +16,7 @@ require (
github.com/charmbracelet/x/ansi v0.11.9
github.com/disintegration/imaging v1.6.2
github.com/fsnotify/fsnotify v1.10.1
github.com/gen2brain/webp v0.6.4
github.com/go-gormigrate/gormigrate/v2 v2.1.7
github.com/go-playground/validator/v10 v10.30.4
github.com/goccy/go-yaml v1.19.2

2
go.sum
View File

@@ -173,6 +173,8 @@ github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx5
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
github.com/gen2brain/webp v0.6.4 h1:SUDdmxADOAiPQ+5ylNmuHhuYf2dOi0KgKZHL5vpVCNU=
github.com/gen2brain/webp v0.6.4/go.mod h1:iGWMaCSw7t3I/Cv9llzEKmpnR36S8lS8VL/ZVjxU0JE=
github.com/go-gormigrate/gormigrate/v2 v2.1.7 h1:PdT4jVPbRb4R+0Ey2R0yJOdctVf4Whiq1Qi4necaZdg=
github.com/go-gormigrate/gormigrate/v2 v2.1.7/go.mod h1:3ouXglTuPrKF5+7cQyVGfvAXTU4vLMaYh9+EPl03uog=
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=

12
modules/lagoon/README.md
View File

@@ -6,11 +6,13 @@ Postgres data layer: the shared GORM connection, per-plugin migrations, model he
`import "git.golem15.com/golem15/summercms/modules/lagoon/attach"`
`import "git.golem15.com/golem15/summercms/modules/lagoon/attach/webp"`
**Name:** Calm, deep water where everything settles; it is the database layer where data is stored.
## Overview
`lagoon` is the WinterCMS models and migrations counterpart: it opens the one `*sql.DB` pool (pgx stdlib driver) that GORM and the rest of the application share, runs each plugin's gormigrate set in its own history table, and ports the Eloquent model conventions that WinterCMS plugins rely on, such as `$fillable`, `$hidden`, `$jsonable`, encrypted casts and Laravel-style validation rules. Its `attach` subpackage ports WinterCMS's `system_files` attachments, storing originals and lazily generated thumbnails in a gocloud.dev blob bucket (a directory, memory, or S3 in binaries built with the `summercms_s3` tag). The application binary gets the migrate and key commands from `lagoon.RuntimeCommands`.
`lagoon` is the WinterCMS models and migrations counterpart: it opens the one `*sql.DB` pool (pgx stdlib driver) that GORM and the rest of the application share, runs each plugin's gormigrate set in its own history table, and ports the Eloquent model conventions that WinterCMS plugins rely on, such as `$fillable`, `$hidden`, `$jsonable`, encrypted casts and Laravel-style validation rules. Its `attach` subpackage ports WinterCMS's `system_files` attachments, storing originals and lazily generated thumbnails in a gocloud.dev blob bucket (a directory, memory, or S3 in binaries built with the `summercms_s3` tag); its `attach/webp` subpackage writes WebP copies of those thumbnails, and only binaries that import it link the encoder. The application binary gets the migrate and key commands from `lagoon.RuntimeCommands`.
## Role in the framework
@@ -39,7 +41,8 @@ Use cases:
- Deferred binding: WinterCMS's `deferred_bindings` table holds the uploads and related-record changes of a form whose record is not saved yet. Every operation takes a `lagoon.DeferredKey` (the form's session key, the owning backend admin's id and the master record's morph type from `lagoon.MorphType`) and never reads or changes another admin's rows, since each row stores `backend_user_id`. `lagoon.DeferredBind` and `lagoon.DeferredUnbind` port WinterCMS's duplicate and cancel rules: a repeated bind writes nothing, and an unbind of a slave with a pending bind deletes that bind and returns it so the caller can remove what it created. `lagoon.DeferredBindings` reads and locks a session's bindings for the save that commits them, `lagoon.DeferredForget` deletes them once applied, and `lagoon.DeferredSlaves` is the subquery a list uses to include pending rows. A child created under deferral carries the `lagoon.DeferredEnvelope` (`{"created":true,"pivot":{...},"upload_id":"..."}`) in `pivot_data`. `lagoon.PurgeDeferred` removes expired bindings: it deletes an unattached `system_files` row a bind points at, and its blobs only after the commit, deletes a child only when its binding carries the created envelope, keeps records that were only linked, and locks each batch with `FOR UPDATE SKIP LOCKED`.
- Logging: GORM's lines no longer go to stdout through GORM's own logger. They arrive through [diary](../diary/README.md) as component `db`: slow queries (over 200 ms) and errors at warn, while a missing record (`gorm.ErrRecordNotFound`) is no longer logged. At verbosity 3 (`-vvv`) every query is traced at info, with placeholders instead of the bound values. lagoon's own warnings log as component `lagoon`, and `attach` as component `attach`.
- Imports from Laravel: `lagoon.DecryptLaravelPayload` decrypts Laravel `encrypted` payloads with the old application key, for one-off data imports.
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), an S3 bucket option behind the `attach.S3BuildTag` build tag (an untagged binary refuses an `s3://` bucket URL at boot with an error naming the tag), public URLs (`attach.PublicURL` for any key, under `storage.uploads.public_url` when a public bucket or CDN serves the files (`attach.PublicURLBase`), `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since WebP cannot be encoded; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name whose extension follows the sniffed content through `attach.DiskExtension`, so a PNG named `x.html` is stored as `.png`, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming, in image mode the `attach.IsAllowedImage` content guard, the refusal of protected uploads while `storage.uploads.public_url` serves the bucket publicly, `attach.ErrProtectedNeedsPrivateBucket`, and `Content-Disposition: attachment` on the blob for every type `attach.IsInlineContentType` does not report as an inline raster image (JPEG, PNG, GIF, WebP, AVIF), while `content_type` keeps the sniffed type), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), an S3 bucket option behind the `attach.S3BuildTag` build tag (an untagged binary refuses an `s3://` bucket URL at boot with an error naming the tag), public URLs (`attach.PublicURL` for any key, under `storage.uploads.public_url` when a public bucket or CDN serves the files (`attach.PublicURLBase`), `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since `attach` itself does not encode WebP; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name whose extension follows the sniffed content through `attach.DiskExtension`, so a PNG named `x.html` is stored as `.png`, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming, in image mode the `attach.IsAllowedImage` content guard, the refusal of protected uploads while `storage.uploads.public_url` serves the bucket publicly, `attach.ErrProtectedNeedsPrivateBucket`, and `Content-Disposition: attachment` on the blob for every type `attach.IsInlineContentType` does not report as an inline raster image (JPEG, PNG, GIF, WebP, AVIF), while `content_type` keeps the sniffed type), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
- WebP variants (`attach/webp`): `webp.Variant` writes a WebP copy of a thumbnail next to it (the thumbnail is made first when missing), at quality 80 (`webp.Quality`), under `attach.VariantKey` (the thumbnail key with the `webp` extension), and returns its key for `attach.PublicURL`. A variant that exists is not written again, and a failed encode leaves no file. `webp.Encode` encodes any image. Callers run it in a background job, not in an upload request. The encoder uses the system libwebp when it is installed (through purego, no cgo) and libwebp transpiled to Go otherwise. A binary that imports the package is dynamically linked against glibc unless it is built with `-tags nodynamic` (transpiled encoder only, static binary).
## Usage
@@ -225,6 +228,10 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
| `attach.HasRelations` | Implemented by an owner model that declares its attachment relations. |
| `attach.BlobKeys` | The original's key and the thumbnail prefix of a file, for `attach.DeleteKeys`. |
| `attach.File.ThumbKey` | Blob key of a lazily generated thumbnail, for serving a protected file's thumbnail without a public URL. |
| `attach.VariantKey` | Blob key of a thumbnail's copy in another format, next to the thumbnail: the extension replaced by the format, or the format appended when the thumbnail already has that extension; empty for a format outside `[a-z0-9]+`. |
| `webp.Encode` | Encodes an image as a lossy WebP at `webp.Quality` (80). |
| `webp.Variant` | Writes the WebP variant of a file's thumbnail at a width, height and mode next to the thumbnail and returns its key; an existing variant is returned unchanged. |
| `webp.Format`, `webp.ContentType` | `webp`, the variant extension, and `image/webp`, its stored content type. |
## Configuration
@@ -270,6 +277,7 @@ An empty `database.dsn` and an unreachable database fail with a `bonfire.WithHin
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [compass](../compass/README.md), [diary](../diary/README.md) (component loggers, GORM's logger), [pact](../pact/README.md), [party](../party/README.md), [phrasebook](../phrasebook/README.md) (validation messages).
- Third-party: `gorm.io/gorm`, `gorm.io/driver/postgres`, `github.com/jackc/pgx/v5` (stdlib driver; `pgconn` for `lagoon.DatabaseLabel`), `github.com/go-gormigrate/gormigrate/v2`, `github.com/go-playground/validator/v10`, `github.com/riverqueue/river` (its `rivermigrate` and `riverdriver/riverdatabasesql` packages, for the River schema in `lagoon.QueueMigrations`).
- Third-party, `attach` only: `gocloud.dev/blob` (file and memory drivers; the S3 driver `gocloud.dev/blob/s3blob` and the AWS SDK v2 only under the `summercms_s3` build tag), `github.com/disintegration/imaging` (thumbnails), `golang.org/x/image/webp` (WebP decoding).
- Third-party, `attach/webp` only: `github.com/gen2brain/webp` (WebP encoding; through it `github.com/ebitengine/purego`), decided in `.planning/notes/2026-10-28-webp-gen2brain.md`.
- Standard library: `database/sql`, `crypto/aes`, `crypto/cipher`, `crypto/hkdf`, `log/slog`, `image`, among others.
## Testing

28
modules/lagoon/attach/variant.go Normal file
View File

@@ -0,0 +1,28 @@
package attach
import (
"path"
"strings"
)
// VariantKey is the blob key of a thumbnail's copy in another image format,
// stored next to the thumbnail: thumbKey with its extension replaced by
// format, so thumb_42_480_0_0_0_auto.jpg becomes
// thumb_42_480_0_0_0_auto.webp. When the thumbnail already carries that
// extension (a thumbnail of a .webp original holds JPEG bytes under the
// original's name, see ThumbKey), format is appended instead
// (thumb_42_480_0_0_0_auto.webp.webp), so the variant never overwrites the
// thumbnail. format follows the token rule of ThumbFilename ([a-z0-9]+,
// lower-cased first); an empty thumbKey or a format outside the rule
// answers "".
func VariantKey(thumbKey, format string) string {
format = strings.ToLower(format)
if thumbKey == "" || !thumbToken.MatchString(format) {
return ""
}
ext := path.Ext(thumbKey)
if strings.EqualFold(strings.TrimPrefix(ext, "."), format) {
return thumbKey + "." + format
}
return strings.TrimSuffix(thumbKey, ext) + "." + format
}

13
modules/lagoon/attach/webp/export_test.go Normal file
View File

@@ -0,0 +1,13 @@
package webp
import (
"image"
"io"
)
// SetEncoderForTest replaces the encoder Variant uses and returns the
// restore function (tests only).
func SetEncoderForTest(fn func(io.Writer, image.Image) error) func() {
encodeImage = fn
return func() { encodeImage = Encode }
}

106
modules/lagoon/attach/webp/webp.go Normal file
View File

@@ -0,0 +1,106 @@
// Package webp encodes WebP images and writes WebP variants of attachment
// thumbnails next to them, through github.com/gen2brain/webp (no cgo: the
// system libwebp through purego when it is installed, else libwebp
// transpiled to Go). It is a separate package so that only binaries that
// import it link the encoder.
package webp
import (
"bytes"
"context"
"errors"
"fmt"
"image"
"io"
"git.golem15.com/golem15/summercms/modules/lagoon/attach"
encoder "github.com/gen2brain/webp"
"gocloud.dev/blob"
"gocloud.dev/gcerrors"
)
// Quality is the encoder quality of Encode, on libwebp's 0 to 100 scale.
const Quality = 80
// Format is the extension of a WebP variant (attach.VariantKey).
const Format = "webp"
// ContentType is the content type a variant is stored with.
const ContentType = "image/webp"
// maxThumbBytes caps the thumbnail Variant reads, like attach caps an
// original.
const maxThumbBytes = 32 << 20
// Encode writes img to w as a lossy WebP at Quality.
func Encode(w io.Writer, img image.Image) error {
if img == nil {
return errors.New("webp: image is nil")
}
return encoder.Encode(w, img, encoder.Options{Quality: Quality})
}
// encodeImage is Encode; tests replace it to fail an encode.
var encodeImage = Encode
// Variant writes the WebP variant of f's thumbnail at w by h in mode (see
// attach.File.ThumbKey, which makes the thumbnail first when it is missing)
// next to that thumbnail, under attach.VariantKey(thumbKey, "webp"), and
// returns the variant's key. A variant that already exists is returned as
// it is, so a second call writes nothing. The thumbnail is encoded in
// memory before anything is written, and the key is deleted when the write
// or its close fails, so a failed encode leaves no partial file. Errors of
// ThumbKey are returned as they are.
func Variant(ctx context.Context, bucket *blob.Bucket, f *attach.File, w, h int, mode string) (string, error) {
thumbKey, err := f.ThumbKey(ctx, bucket, w, h, mode)
if err != nil {
return "", err
}
key := attach.VariantKey(thumbKey, Format)
if key == "" {
return "", fmt.Errorf("webp: no variant key for %q", thumbKey)
}
exists, err := bucket.Exists(ctx, key)
if err != nil {
return "", fmt.Errorf("webp: variant exists: %w", err)
}
if exists {
return key, nil
}
r, err := bucket.NewReader(ctx, thumbKey, nil)
if err != nil {
return "", fmt.Errorf("webp: read thumbnail: %w", err)
}
raw, err := io.ReadAll(io.LimitReader(r, maxThumbBytes))
closeErr := r.Close()
if err != nil {
return "", fmt.Errorf("webp: read thumbnail: %w", err)
}
if closeErr != nil {
return "", fmt.Errorf("webp: read thumbnail: %w", closeErr)
}
img, _, err := image.Decode(bytes.NewReader(raw))
if err != nil {
return "", fmt.Errorf("webp: decode thumbnail %q: %w", thumbKey, err)
}
var buf bytes.Buffer
if err := encodeImage(&buf, img); err != nil {
return "", fmt.Errorf("webp: encode %q: %w", thumbKey, err)
}
wr, err := bucket.NewWriter(ctx, key, &blob.WriterOptions{ContentType: ContentType})
if err != nil {
return "", fmt.Errorf("webp: variant writer: %w", err)
}
_, writeErr := wr.Write(buf.Bytes())
closeErr = wr.Close()
if writeErr != nil || closeErr != nil {
if delErr := bucket.Delete(ctx, key); delErr != nil && gcerrors.Code(delErr) != gcerrors.NotFound {
return "", fmt.Errorf("webp: delete partial variant %q: %w", key, delErr)
}
if writeErr != nil {
return "", fmt.Errorf("webp: write variant: %w", writeErr)
}
return "", fmt.Errorf("webp: write variant: %w", closeErr)
}
return key, nil
}

203
modules/lagoon/attach/webp/webp_smoke_test.go Normal file
View File

@@ -0,0 +1,203 @@
package webp_test
import (
"bytes"
"context"
"errors"
"fmt"
"image"
"image/color"
"image/jpeg"
"io"
"testing"
"git.golem15.com/golem15/summercms/modules/lagoon/attach"
"git.golem15.com/golem15/summercms/modules/lagoon/attach/webp"
"gocloud.dev/blob"
"gocloud.dev/blob/memblob"
)
// gradient is a w by h test picture.
func gradient(w, h int) *image.RGBA {
img := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.Set(x, y, color.RGBA{uint8(x * 255 / w), uint8(y * 255 / h), 128, 255})
}
}
return img
}
// isWebP reports whether raw starts with a RIFF/WEBP header.
func isWebP(raw []byte) bool {
return len(raw) > 12 && string(raw[0:4]) == "RIFF" && string(raw[8:12]) == "WEBP"
}
// putOriginal stores a JPEG original for f in bucket.
func putOriginal(t *testing.T, bucket *blob.Bucket, f *attach.File, w, h int) {
t.Helper()
var buf bytes.Buffer
if err := jpeg.Encode(&buf, gradient(w, h), nil); err != nil {
t.Fatal(err)
}
if err := bucket.WriteAll(t.Context(), attach.BlobKey(f.DiskName), buf.Bytes(), nil); err != nil {
t.Fatal(err)
}
}
func TestEncodeWritesWebP(t *testing.T) {
var buf bytes.Buffer
if err := webp.Encode(&buf, gradient(64, 40)); err != nil {
t.Fatal(err)
}
if !isWebP(buf.Bytes()) {
t.Fatalf("Encode wrote % x, want a RIFF/WEBP header", buf.Bytes()[:16])
}
cfg, format, err := image.DecodeConfig(bytes.NewReader(buf.Bytes()))
if err != nil || format != "webp" || cfg.Width != 64 || cfg.Height != 40 {
t.Fatalf("decoded config %+v %q err=%v, want a 64x40 webp", cfg, format, err)
}
if err := webp.Encode(&buf, nil); err == nil {
t.Fatal("webp.Encode(nil) must fail")
}
}
func TestVariantKey(t *testing.T) {
for _, tc := range []struct{ thumb, format, want string }{
{"abc/123/xyz/thumb_42_480_0_0_0_auto.jpg", "webp", "abc/123/xyz/thumb_42_480_0_0_0_auto.webp"},
{"abc/123/xyz/thumb_42_480_0_0_0_auto.png", "WEBP", "abc/123/xyz/thumb_42_480_0_0_0_auto.webp"},
{"abc/123/xyz/thumb_42_480_0_0_0_auto.webp", "webp", "abc/123/xyz/thumb_42_480_0_0_0_auto.webp.webp"},
{"abc/123/xyz/thumb_42_480_0_0_0_auto.jpg", "../x", ""},
{"abc/123/xyz/thumb_42_480_0_0_0_auto.jpg", "", ""},
{"", "webp", ""},
} {
if got := attach.VariantKey(tc.thumb, tc.format); got != tc.want {
t.Errorf("VariantKey(%q, %q) = %q, want %q", tc.thumb, tc.format, got, tc.want)
}
}
}
func TestVariantWritesOnceNextToTheThumbnail(t *testing.T) {
ctx := t.Context()
bucket := memblob.OpenBucket(nil)
defer bucket.Close()
f := &attach.File{ID: 5, DiskName: "abc123xyz.jpg", FileName: "cover.jpg"}
putOriginal(t, bucket, f, 400, 300)
key, err := webp.Variant(ctx, bucket, f, 120, 90, "auto")
if err != nil {
t.Fatal(err)
}
if want := "abc/123/xyz/thumb_5_120_90_0_0_auto.webp"; key != want {
t.Fatalf("variant key = %q, want %q", key, want)
}
if ok, err := bucket.Exists(ctx, "abc/123/xyz/thumb_5_120_90_0_0_auto.jpg"); err != nil || !ok {
t.Fatalf("the thumbnail was not made first: %v %v", ok, err)
}
raw, err := bucket.ReadAll(ctx, key)
if err != nil || !isWebP(raw) {
t.Fatalf("variant bytes = % x err=%v, want WebP", raw[:min(16, len(raw))], err)
}
attrs, err := bucket.Attributes(ctx, key)
if err != nil || attrs.ContentType != webp.ContentType {
t.Fatalf("variant content type = %v err=%v", attrs, err)
}
cfg, _, err := image.DecodeConfig(bytes.NewReader(raw))
if err != nil || cfg.Width != 120 || cfg.Height != 90 {
t.Fatalf("variant is %dx%d err=%v, want the thumbnail's 120x90", cfg.Width, cfg.Height, err)
}
// A second call finds the variant and writes nothing.
if err := bucket.WriteAll(ctx, key, []byte("kept"), nil); err != nil {
t.Fatal(err)
}
again, err := webp.Variant(ctx, bucket, f, 120, 90, "auto")
if err != nil || again != key {
t.Fatalf("second Variant = %q err=%v", again, err)
}
if raw, _ := bucket.ReadAll(ctx, key); string(raw) != "kept" {
t.Fatalf("second Variant rewrote the variant: % x", raw[:min(16, len(raw))])
}
}
func TestVariantOfAWebPOriginalKeepsItsThumbnail(t *testing.T) {
ctx := t.Context()
bucket := memblob.OpenBucket(nil)
defer bucket.Close()
f := &attach.File{ID: 9, DiskName: "def456uvw.webp"}
var buf bytes.Buffer
if err := webp.Encode(&buf, gradient(200, 100)); err != nil {
t.Fatal(err)
}
if err := bucket.WriteAll(ctx, attach.BlobKey(f.DiskName), buf.Bytes(), nil); err != nil {
t.Fatal(err)
}
key, err := webp.Variant(ctx, bucket, f, 100, 50, "auto")
if err != nil {
t.Fatal(err)
}
thumb := "def/456/uvw/thumb_9_100_50_0_0_auto.webp"
if key != thumb+".webp" {
t.Fatalf("variant key = %q, want %q", key, thumb+".webp")
}
if raw, _ := bucket.ReadAll(ctx, thumb); isWebP(raw) {
t.Fatal("the thumbnail of a webp original was overwritten with the variant")
}
if raw, _ := bucket.ReadAll(ctx, key); !isWebP(raw) {
t.Fatal("the variant is not WebP")
}
}
func TestVariantEncodeFailureLeavesNoFile(t *testing.T) {
ctx := t.Context()
bucket := memblob.OpenBucket(nil)
defer bucket.Close()
f := &attach.File{ID: 7, DiskName: "ghi789rst.jpg"}
putOriginal(t, bucket, f, 300, 200)
boom := errors.New("boom")
restore := webp.SetEncoderForTest(func(w io.Writer, _ image.Image) error {
_, _ = w.Write([]byte("RIFF partial"))
return boom
})
t.Cleanup(restore)
if _, err := webp.Variant(ctx, bucket, f, 60, 40, "auto"); !errors.Is(err, boom) {
t.Fatalf("Variant err = %v, want the encode failure", err)
}
if ok, err := bucket.Exists(ctx, "ghi/789/rst/thumb_7_60_40_0_0_auto.webp"); err != nil || ok {
t.Fatalf("a failed encode left a variant: %v %v", ok, err)
}
restore()
if _, err := webp.Variant(ctx, bucket, f, 60, 40, "auto"); err != nil {
t.Fatalf("the next run after a failure: %v", err)
}
}
func ExampleVariant() {
ctx := context.Background()
bucket := memblob.OpenBucket(nil)
defer bucket.Close()
// The system_files row of a cover image and its stored original.
f := &attach.File{ID: 42, DiskName: "5f1d0c2e9a7b4c3d8e6f.jpg", FileName: "cover.jpg"}
var img bytes.Buffer
if err := jpeg.Encode(&img, image.NewRGBA(image.Rect(0, 0, 1600, 1200)), nil); err != nil {
fmt.Println(err)
return
}
if err := bucket.WriteAll(ctx, attach.BlobKey(f.DiskName), img.Bytes(), nil); err != nil {
fmt.Println(err)
return
}
// In a background job: the 800 px thumbnail and its WebP copy.
key, err := webp.Variant(ctx, bucket, f, 800, 800, "auto")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(key)
fmt.Println(attach.PublicURL(key))
// Output:
// 5f1/d0c/2e9/thumb_42_800_800_0_0_auto.webp
// /storage/uploads/5f1/d0c/2e9/thumb_42_800_800_0_0_auto.webp
}