feat(attach): absolute public URL from config

- attach.AbsolutePublicURL builds the absolute public URL of a blob key from
  storage.uploads.public_url, or from app.url and the public path prefix
- it reads the config only, not the values OpenBucket recorded, so a job
  worker that opened no bucket gets the same URL as serve
- attach.ErrNoAbsoluteURL when neither base is configured

Plan 35-01, task 5.
This commit is contained in:
Jakub Zych
2026-11-13 11:01:43 +01:00
parent f8fa0f1b4b
commit 935e4bf64f
3 changed files with 62 additions and 1 deletions

View File

@@ -41,7 +41,7 @@ Use cases:
- Deferred binding: WinterCMS's `deferred_bindings` table holds the uploads and related-record changes of a form whose record is not saved yet. Every operation takes a `lagoon.DeferredKey` (the form's session key, the owning backend admin's id and the master record's morph type from `lagoon.MorphType`) and never reads or changes another admin's rows, since each row stores `backend_user_id`. `lagoon.DeferredBind` and `lagoon.DeferredUnbind` port WinterCMS's duplicate and cancel rules: a repeated bind writes nothing, and an unbind of a slave with a pending bind deletes that bind and returns it so the caller can remove what it created. `lagoon.DeferredBindings` reads and locks a session's bindings for the save that commits them, `lagoon.DeferredForget` deletes them once applied, and `lagoon.DeferredSlaves` is the subquery a list uses to include pending rows. A child created under deferral carries the `lagoon.DeferredEnvelope` (`{"created":true,"pivot":{...},"upload_id":"..."}`) in `pivot_data`. `lagoon.PurgeDeferred` removes expired bindings: it deletes an unattached `system_files` row a bind points at, and its blobs only after the commit, deletes a child only when its binding carries the created envelope, keeps records that were only linked, and locks each batch with `FOR UPDATE SKIP LOCKED`.
- Logging: GORM's lines no longer go to stdout through GORM's own logger. They arrive through [diary](../diary/README.md) as component `db`: slow queries (over 200 ms) and errors at warn, while a missing record (`gorm.ErrRecordNotFound`) is no longer logged. At verbosity 3 (`-vvv`) every query is traced at info, with placeholders instead of the bound values. lagoon's own warnings log as component `lagoon`, and `attach` as component `attach`.
- Imports from Laravel: `lagoon.DecryptLaravelPayload` decrypts Laravel `encrypted` payloads with the old application key, for one-off data imports.
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), an S3 bucket option behind the `attach.S3BuildTag` build tag (an untagged binary refuses an `s3://` bucket URL at boot with an error naming the tag), public URLs (`attach.PublicURL` for any key, under `storage.uploads.public_url` when a public bucket or CDN serves the files (`attach.PublicURLBase`), `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since `attach` itself does not encode WebP; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name whose extension follows the sniffed content through `attach.DiskExtension`, so a PNG named `x.html` is stored as `.png`, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming, in image mode the `attach.IsAllowedImage` content guard, the refusal of protected uploads while `storage.uploads.public_url` serves the bucket publicly, `attach.ErrProtectedNeedsPrivateBucket`, and `Content-Disposition: attachment` on the blob for every type `attach.IsInlineContentType` does not report as an inline raster image (JPEG, PNG, GIF, WebP, AVIF), while `content_type` keeps the sniffed type; `attach.UploadWriterOptions` gives code which writes an upload to the bucket without `attach.Store` the same content type and disposition), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving (`attach.StaticHandler`, with an optional `is_public` gate in `attach.StaticHandlerPublic`) that sends each file's stored Content-Type and Content-Disposition with `X-Content-Type-Options: nosniff`, and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), an S3 bucket option behind the `attach.S3BuildTag` build tag (an untagged binary refuses an `s3://` bucket URL at boot with an error naming the tag), public URLs (`attach.PublicURL` for any key, under `storage.uploads.public_url` when a public bucket or CDN serves the files (`attach.PublicURLBase`), `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout, and `attach.AbsolutePublicURL` for an absolute URL built from configuration alone, for mail and other use outside the site), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since `attach` itself does not encode WebP; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name whose extension follows the sniffed content through `attach.DiskExtension`, so a PNG named `x.html` is stored as `.png`, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming, in image mode the `attach.IsAllowedImage` content guard, the refusal of protected uploads while `storage.uploads.public_url` serves the bucket publicly, `attach.ErrProtectedNeedsPrivateBucket`, and `Content-Disposition: attachment` on the blob for every type `attach.IsInlineContentType` does not report as an inline raster image (JPEG, PNG, GIF, WebP, AVIF), while `content_type` keeps the sniffed type; `attach.UploadWriterOptions` gives code which writes an upload to the bucket without `attach.Store` the same content type and disposition), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving (`attach.StaticHandler`, with an optional `is_public` gate in `attach.StaticHandlerPublic`) that sends each file's stored Content-Type and Content-Disposition with `X-Content-Type-Options: nosniff`, and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
- WebP variants (`attach/webp`): `webp.Variant` writes a WebP copy of a thumbnail next to it (the thumbnail is made first when missing), at quality 80 (`webp.Quality`), under `attach.VariantKey` (the thumbnail key with the `webp` extension), and returns its key for `attach.PublicURL`. A variant that exists is not written again, and a failed encode leaves no file. `webp.Encode` encodes any image. Callers run it in a background job, not in an upload request. The encoder uses the system libwebp when it is installed (through purego, no cgo) and libwebp transpiled to Go otherwise. A binary that imports the package is dynamically linked against glibc unless it is built with `-tags nodynamic` (transpiled encoder only, static binary). Importing the package also registers the encoder library's WebP decoder ahead of the pure Go one for the whole process, so every WebP decode, uploads included (thumbnails, size checks), goes through the system libwebp when it is installed: keep libwebp patched, or build with `-tags nodynamic`, which decodes with the transpiled copy.
## Usage
@@ -199,6 +199,8 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
| `attach.File` | The `system_files` row model; `attach.File.URL` is the public URL of the original. |
| `attach.PublicURL` | Public URL of a blob key: under `storage.uploads.public_url` when set, else under `storage.uploads.public_path_prefix`. |
| `attach.PublicURLBase` | `storage.uploads.public_url` as recorded by `attach.OpenBucket`, empty when unset. |
| `attach.AbsolutePublicURL` | Absolute public URL of a blob key from a `*compass.Config`: `storage.uploads.public_url` and the key, else `app.url`, `storage.uploads.public_path_prefix` and the key. It reads the config only, so it works in a process that opened no bucket. |
| `attach.ErrNoAbsoluteURL` | Returned by `attach.AbsolutePublicURL` when neither `storage.uploads.public_url` nor an absolute `http` or `https` `app.url` is set. |
| `attach.BrokenImagePNG` | WinterCMS's broken-image picture, stored as the thumbnail of an unusable original. |
| `attach.Owner` | Implemented by models that own attachments; returns the stored morph type name. |
| `attach.OpenBucket` | Opens the uploads bucket from config; refuses `s3://` in a binary built without `attach.S3BuildTag`. |

View File

@@ -0,0 +1,50 @@
package attach
import (
"errors"
"net/url"
"strings"
"git.golem15.com/golem15/summercms/modules/compass"
)
// ErrNoAbsoluteURL is returned by AbsolutePublicURL when the configuration
// gives no absolute base: neither storage.uploads.public_url nor an absolute
// app.url is set.
var ErrNoAbsoluteURL = errors.New("attach: no absolute public URL: set storage.uploads.public_url or app.url")
// AbsolutePublicURL returns the absolute public URL of a blob key, for use
// outside the site, as in mail. With storage.uploads.public_url set it is
// that base and the key joined by one slash, as PublicURL returns. Otherwise
// it is app.url, storage.uploads.public_path_prefix (the default prefix when
// unset) and the key. With neither it returns ErrNoAbsoluteURL; a malformed
// public_url is its own error.
//
// It reads cfg only, never the values OpenBucket recorded, so it returns the
// same URL in a process that opened no bucket, such as a job worker.
func AbsolutePublicURL(cfg *compass.Config, key string) (string, error) {
if cfg == nil {
return "", ErrNoAbsoluteURL
}
key = strings.TrimLeft(key, "/")
base, err := parsePublicURL(cfg.String("storage.uploads.public_url"))
if err != nil {
return "", err
}
if base != nil {
return strings.TrimRight(base.String(), "/") + "/" + key, nil
}
app, err := url.Parse(strings.TrimSpace(cfg.String("app.url")))
if err != nil || app.Host == "" || (!strings.EqualFold(app.Scheme, "http") && !strings.EqualFold(app.Scheme, "https")) {
return "", ErrNoAbsoluteURL
}
prefix := strings.TrimSpace(cfg.String("storage.uploads.public_path_prefix"))
if prefix == "" {
prefix = defaultPublicPathPrefix
}
out := app.Scheme + "://" + app.Host + strings.TrimRight(app.Path, "/")
if prefix = strings.Trim(prefix, "/"); prefix != "" {
out += "/" + prefix
}
return out + "/" + key, nil
}