feat(lagoon/attach): export UploadWriterOptions for blob writers that bypass Store (quick-261106-d20)

- attach.UploadWriterOptions (was the unexported uploadWriterOptions) gives
  code that writes an upload to the bucket itself the same Content-Type and
  Content-Disposition: attachment for non-inline types that attach.Store stores
- writeBlob calls it; Store behaviour unchanged
- TestUploadWriterOptions table test
- lagoon README, docs/database/attachments.md and docs/services/storage.md
  describe it for writers that bypass attach.Store

docs/database/attachments.md

@@ -139,6 +139,8 @@ The disk name's extension follows the content, not the client's name, because a
 
 `attach.Store` writes every file whose type `attach.IsInlineContentType` does not report as a raster image (JPEG, PNG, GIF, WebP, AVIF) with `Content-Disposition: attachment`, so a browser that opens it from a public bucket or through `attach.StaticHandler` and `attach.StaticHandlerPublic` downloads it instead of rendering it. `<img>`, `<video>` and `<audio>` embeds and the `content_type` column are unaffected, and thumbnails stay inline.
 
+Code that writes an upload to the bucket itself instead of calling `attach.Store` gets the same protection the same way: sniff the type first, name the file with `attach.DiskExtension(contentType, clientExt)` and pass `attach.UploadWriterOptions(contentType)` to the bucket writer (`NewWriter` or `WriteAll`). `file_name` keeps the client's name.
+
 The content type is sniffed from the first bytes of the body, never taken from the client. `Limits.MIMETypes` narrows it further: an entry with a slash is a MIME pattern such as `image/png` or `image/*`, an entry without one is an extension, and a file that matches no entry returns `attach.ErrMIMEType`. `Limits.MaxBytes` is enforced while the body streams into the bucket; a body one byte over the limit returns `attach.ErrTooLarge` and leaves no blob and no row behind.
 
 With `Limits.Image` set, the content must pass `attach.IsAllowedImage`: the bytes must sniff as one of `attach.AllowedImageMIMEs` (JPEG, PNG, GIF or WebP), the header must decode as that format, and the image may not exceed `attach.MaxImagePixels` (4096 by 4096), so every accepted image can be thumbnailed. Anything else, an SVG for example, returns `attach.ErrNotImage`.

docs/services/storage.md

@@ -159,7 +159,7 @@ Extend the extension list only with types the application accepts (see `attach.D
 
 **A public S3 bucket, or a CDN that pulls from it, serving the files under `storage.uploads.public_url`.** It does not look at the extension: it sends the headers stored with the object. The stored Content-Type is the sniffed type, so an HTML body uploaded as `note.txt` (txt is in `attach.DefaultFileExtensions`) is stored as `<hex>.txt` with `text/html`. `attach.Store` therefore also stores `Content-Disposition: attachment` with every object whose type `attach.IsInlineContentType` does not report as a raster image (JPEG, PNG, GIF, WebP, AVIF). S3, and a CDN that forwards the stored headers, make a browser that opens the URL download HTML, SVG, PDF or plain text instead of rendering it. `<img>`, `<video>` and `<audio>` embeds are not affected, and thumbnails, being images, carry no disposition.
 
-Objects written before this change, or by code that bypasses `attach.Store`, carry no disposition: set it on them, or add the rule in the CDN. `attach.StaticHandler` and `attach.StaticHandlerPublic` send the stored disposition too, so the same caveat applies to them, while the built-in route also decides from the type. A CDN that rewrites or drops the origin's headers needs the rule itself. Either way, serve such a bucket from its own origin, never the application's, and set `X-Content-Type-Options: nosniff` through the bucket's or the CDN's response-header settings, since S3 cannot store that header on an object.
+Code that bypasses `attach.Store` and writes uploads to the bucket itself should write with `attach.UploadWriterOptions` and name its files with `attach.DiskExtension`, which gives its objects the same type and disposition. Objects written before this change, or by code that does neither, carry no disposition: set it on them, or add the rule in the CDN. `attach.StaticHandler` and `attach.StaticHandlerPublic` send the stored disposition too, so the same caveat applies to them, while the built-in route also decides from the type. A CDN that rewrites or drops the origin's headers needs the rule itself. Either way, serve such a bucket from its own origin, never the application's, and set `X-Content-Type-Options: nosniff` through the bucket's or the CDN's response-header settings, since S3 cannot store that header on an object.
 
 A `file://` bucket keeps the disposition in the file driver's `.attrs` sidecar files. `attach.StaticHandler`, `attach.StaticHandlerPublic` and the built-in route read the sidecar; a static server does not, so the static-server case above applies only to a web server reading the directory.
 

modules/lagoon/README.md

@@ -41,7 +41,7 @@ Use cases:
 - Deferred binding: WinterCMS's `deferred_bindings` table holds the uploads and related-record changes of a form whose record is not saved yet. Every operation takes a `lagoon.DeferredKey` (the form's session key, the owning backend admin's id and the master record's morph type from `lagoon.MorphType`) and never reads or changes another admin's rows, since each row stores `backend_user_id`. `lagoon.DeferredBind` and `lagoon.DeferredUnbind` port WinterCMS's duplicate and cancel rules: a repeated bind writes nothing, and an unbind of a slave with a pending bind deletes that bind and returns it so the caller can remove what it created. `lagoon.DeferredBindings` reads and locks a session's bindings for the save that commits them, `lagoon.DeferredForget` deletes them once applied, and `lagoon.DeferredSlaves` is the subquery a list uses to include pending rows. A child created under deferral carries the `lagoon.DeferredEnvelope` (`{"created":true,"pivot":{...},"upload_id":"..."}`) in `pivot_data`. `lagoon.PurgeDeferred` removes expired bindings: it deletes an unattached `system_files` row a bind points at, and its blobs only after the commit, deletes a child only when its binding carries the created envelope, keeps records that were only linked, and locks each batch with `FOR UPDATE SKIP LOCKED`.
 - Logging: GORM's lines no longer go to stdout through GORM's own logger. They arrive through [diary](../diary/README.md) as component `db`: slow queries (over 200 ms) and errors at warn, while a missing record (`gorm.ErrRecordNotFound`) is no longer logged. At verbosity 3 (`-vvv`) every query is traced at info, with placeholders instead of the bound values. lagoon's own warnings log as component `lagoon`, and `attach` as component `attach`.
 - Imports from Laravel: `lagoon.DecryptLaravelPayload` decrypts Laravel `encrypted` payloads with the old application key, for one-off data imports.
-- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), an S3 bucket option behind the `attach.S3BuildTag` build tag (an untagged binary refuses an `s3://` bucket URL at boot with an error naming the tag), public URLs (`attach.PublicURL` for any key, under `storage.uploads.public_url` when a public bucket or CDN serves the files (`attach.PublicURLBase`), `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since `attach` itself does not encode WebP; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name whose extension follows the sniffed content through `attach.DiskExtension`, so a PNG named `x.html` is stored as `.png`, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming, in image mode the `attach.IsAllowedImage` content guard, the refusal of protected uploads while `storage.uploads.public_url` serves the bucket publicly, `attach.ErrProtectedNeedsPrivateBucket`, and `Content-Disposition: attachment` on the blob for every type `attach.IsInlineContentType` does not report as an inline raster image (JPEG, PNG, GIF, WebP, AVIF), while `content_type` keeps the sniffed type), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving (`attach.StaticHandler`, with an optional `is_public` gate in `attach.StaticHandlerPublic`) that sends each file's stored Content-Type and Content-Disposition with `X-Content-Type-Options: nosniff`, and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
+- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), an S3 bucket option behind the `attach.S3BuildTag` build tag (an untagged binary refuses an `s3://` bucket URL at boot with an error naming the tag), public URLs (`attach.PublicURL` for any key, under `storage.uploads.public_url` when a public bucket or CDN serves the files (`attach.PublicURLBase`), `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since `attach` itself does not encode WebP; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name whose extension follows the sniffed content through `attach.DiskExtension`, so a PNG named `x.html` is stored as `.png`, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming, in image mode the `attach.IsAllowedImage` content guard, the refusal of protected uploads while `storage.uploads.public_url` serves the bucket publicly, `attach.ErrProtectedNeedsPrivateBucket`, and `Content-Disposition: attachment` on the blob for every type `attach.IsInlineContentType` does not report as an inline raster image (JPEG, PNG, GIF, WebP, AVIF), while `content_type` keeps the sniffed type; `attach.UploadWriterOptions` gives code which writes an upload to the bucket without `attach.Store` the same content type and disposition), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving (`attach.StaticHandler`, with an optional `is_public` gate in `attach.StaticHandlerPublic`) that sends each file's stored Content-Type and Content-Disposition with `X-Content-Type-Options: nosniff`, and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
 - WebP variants (`attach/webp`): `webp.Variant` writes a WebP copy of a thumbnail next to it (the thumbnail is made first when missing), at quality 80 (`webp.Quality`), under `attach.VariantKey` (the thumbnail key with the `webp` extension), and returns its key for `attach.PublicURL`. A variant that exists is not written again, and a failed encode leaves no file. `webp.Encode` encodes any image. Callers run it in a background job, not in an upload request. The encoder uses the system libwebp when it is installed (through purego, no cgo) and libwebp transpiled to Go otherwise. A binary that imports the package is dynamically linked against glibc unless it is built with `-tags nodynamic` (transpiled encoder only, static binary). Importing the package also registers the encoder library's WebP decoder ahead of the pure Go one for the whole process, so every WebP decode, uploads included (thumbnails, size checks), goes through the system libwebp when it is installed: keep libwebp patched, or build with `-tags nodynamic`, which decodes with the transpiled copy.
 
 ## Usage
@@ -223,6 +223,7 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
 | `attach.AllowedImageMIMEs` | The sniffed content types the image guard accepts. |
 | `attach.IsAllowedImage` | The image guard: sniffed type, decoded header and a pixel ceiling, failing closed. |
 | `attach.IsInlineContentType` | Reports whether a content type, parameters ignored, is a raster image a browser may show inline (JPEG, PNG, GIF, WebP, AVIF); `attach.Store` stores every other type with `Content-Disposition: attachment`, and the surf uploads route serves it as a download. |
+| `attach.UploadWriterOptions` | The blob writer options of an upload: the content type plus `Content-Disposition: attachment` for a type `attach.IsInlineContentType` does not report as inline. `attach.Store` writes with the same options; pair it with `attach.DiskExtension` when writing uploads yourself. |
 | `attach.MaxImagePixels` | The image guard's pixel ceiling, 4096 by 4096. |
 | `attach.Relation` | One attachOne or attachMany relation: `Name`, `Many` and `Public`. |
 | `attach.HasRelations` | Implemented by an owner model that declares its attachment relations. |

modules/lagoon/attach/inline.go

@@ -33,9 +33,14 @@ func IsInlineContentType(contentType string) bool {
 	return err == nil && inlineContentTypes[strings.ToLower(mediaType)]
 }
 
-// uploadWriterOptions are the blob writer options of an upload: the content
-// type, plus the attachment disposition when the type is not inline.
-func uploadWriterOptions(contentType string) *blob.WriterOptions {
+// UploadWriterOptions returns the blob writer options of an upload: the
+// content type, plus Content-Disposition: attachment (no filename parameter)
+// for every type IsInlineContentType does not report as inline. Store writes
+// with these options. Code that writes an upload to the bucket itself, with
+// NewWriter or WriteAll, passes the sniffed type (the value stored in
+// content_type) and names the file with DiskExtension, so its blob is served
+// like one Store wrote. Each call returns a new value the caller may extend.
+func UploadWriterOptions(contentType string) *blob.WriterOptions {
 	opts := &blob.WriterOptions{ContentType: contentType}
 	if !IsInlineContentType(contentType) {
 		opts.ContentDisposition = attachmentDisposition

modules/lagoon/attach/inline_test.go

@@ -1,6 +1,7 @@
 package attach
 
 import (
+	"reflect"
 	"strings"
 	"testing"
 
@@ -40,6 +41,44 @@ func TestIsInlineContentType(t *testing.T) {
 	}
 }
 
+func TestUploadWriterOptions(t *testing.T) {
+	cases := []struct {
+		in          string
+		disposition string
+	}{
+		{"image/png", ""},
+		{"image/jpeg", ""},
+		{"image/gif", ""},
+		{"image/avif", ""},
+		{"IMAGE/WEBP; charset=binary", ""},
+		{"text/html", "attachment"},
+		{"text/xml", "attachment"},
+		{"image/svg+xml", "attachment"},
+		{"application/pdf", "attachment"},
+		{"text/plain", "attachment"},
+		{"application/octet-stream", "attachment"},
+		{"", "attachment"},
+	}
+	for _, tc := range cases {
+		got := UploadWriterOptions(tc.in)
+		want := &blob.WriterOptions{ContentType: tc.in, ContentDisposition: tc.disposition}
+		if !reflect.DeepEqual(got, want) {
+			t.Errorf("UploadWriterOptions(%q) = %+v, want %+v", tc.in, got, want)
+		}
+	}
+
+	first := UploadWriterOptions("text/html")
+	second := UploadWriterOptions("text/html")
+	if first == second {
+		t.Fatal("UploadWriterOptions returned the same pointer twice")
+	}
+	first.ContentDisposition = "inline"
+	first.CacheControl = "no-store"
+	if third := UploadWriterOptions("text/html"); third.ContentDisposition != "attachment" || third.CacheControl != "" {
+		t.Errorf("mutating one result changed the next: %+v", third)
+	}
+}
+
 // TestWriteBlobDisposition is the driver half of the public-bucket fix: a
 // public bucket serves the headers Store stores with the object, so the
 // attachment disposition must survive the write on the real drivers. The

modules/lagoon/attach/store.go

@@ -205,7 +205,7 @@ func Store(ctx context.Context, db *gorm.DB, bucket *blob.Bucket, in Upload, lim
 func writeBlob(ctx context.Context, bucket *blob.Bucket, key string, r io.Reader, contentType string, limit int64) (int64, error) {
 	writeCtx, cancel := context.WithCancel(ctx)
 	defer cancel()
-	w, err := bucket.NewWriter(writeCtx, key, uploadWriterOptions(contentType))
+	w, err := bucket.NewWriter(writeCtx, key, UploadWriterOptions(contentType))
 	if err != nil {
 		return 0, fmt.Errorf("attach: blob writer: %w", err)
 	}