Files
summercms/.planning/phases/05-data-layer-full-fidelity/05-04-PLAN.md
2026-09-18 17:31:53 +02:00

22 KiB

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
phase plan type wave depends_on files_modified autonomous requirements must_haves
05-data-layer-full-fidelity 04 execute 3
05-02
summercms.go/go.mod
summercms.go/go.sum
summercms.go/lagoon/attach/file.go
summercms.go/lagoon/attach/file_test.go
summercms.go/lagoon/attach/migrations.go
summercms.go/lagoon/attach/migrations_test.go
summercms.go/lagoon/attach/bucket.go
summercms.go/lagoon/attach/thumb.go
summercms.go/lagoon/attach/thumb_test.go
summercms.go/lagoon/attach/static.go
summercms.go/lagoon/attach/static_test.go
summercms.go/lagoon/migrations.go
summercms.go/lagoon/migrations_test.go
fonoteka.go/config/storage.yaml
fonoteka.go/plugins/golem15/fonoteka/models/album.go
fonoteka.go/plugins/golem15/fonoteka/models/collection.go
true
DATA-08
DATA-09
truths artifacts key_links
The framework-owned system_files table (Winter's exact shape and column names) migrates before every plugin's own migration set runs, via lagoon.Migrate itself, not an app-level wiring call (D-14)
Thumb(w,h,mode) reproduces Winter's exact filename (thumb_<id>_<w>_<h>_0_0_<mode>.<ext>) and partition-directory rule, generating lazily through gocloud.dev/blob on first call via disintegration/imaging (D-15, D-16, D-17, D-18)
Files are stored/deleted through gocloud.dev/blob (fileblob in dev/prod rooted at storage/app/uploads, memblob in unit tests), never direct os.WriteFile/ReadFile (D-15, D-16)
Soft-deleting an attachment's owner keeps system_files rows and blobs; force-deleting removes the row inside the delete transaction and the blob (plus thumbs) only after commit (D-19)
Album and Collection carry per-model morph names that keep the PHP class string in attachment_type, and Collection's photos are ordered while its image is a single attachOne (Collection.php ~91-107)
path provides
summercms.go/lagoon/attach/file.go File model (system_files) + Owner interface (MorphName)
path provides
summercms.go/lagoon/attach/migrations.go create_system_files migration, run before every plugin set
path provides
summercms.go/lagoon/attach/thumb.go Thumb(w,h,mode) naming + lazy disintegration/imaging resize
path provides
summercms.go/lagoon/attach/static.go StaticHandler(bucket, prefix) http.Handler for public files (not yet route-wired — Phase 6 territory)
from to via pattern
summercms.go/lagoon/migrations.go summercms.go/lagoon/attach/migrations.go Migrate() runs attach's migration set first, before iterating plugins attach.Migrate(|attach.Migrations
from to via pattern
fonoteka.go/plugins/golem15/fonoteka/models/album.go summercms.go/lagoon/attach/file.go Album implements attach.Owner via MorphName() returning the PHP class string func (Album) MorphName() string
Ship the framework-owned `system_files` attachment table and `File` model, the `gocloud.dev/blob` bucket wiring (fileblob/memblob), Winter-exact `Thumb()` naming with lazy `disintegration/imaging` resize, the delete-lifecycle rule (soft-delete keeps blobs, force-delete removes them after commit), and wire `Album`'s ordered photos and `Collection`'s photos/image onto it — the full D-15 storage scope the user chose over the smaller table-only cut.

Purpose: this is the phase's largest single new subsystem (first blob-storage code in the repo) and the plan most likely to blow context if under-scoped — it ships storage end to end (metadata + bytes + thumbnails + lifecycle) but stops at the HTTP upload endpoint, which stays out of scope until Phase 12 per the phase boundary. Output: lagoon/attach package (File, migrations, bucket, thumb, static handler); disintegration/imaging and gocloud.dev/blob added to summercms.go/go.mod; Album/Collection attachment wiring in fonoteka.go.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/phases/05-data-layer-full-fidelity/05-CONTEXT.md @.planning/phases/05-data-layer-full-fidelity/05-RESEARCH.md @.planning/phases/05-data-layer-full-fidelity/05-02-SUMMARY.md

From summercms.go/lagoon/lifecycle.go (Plan 05-01):

type HasAfterDelete interface { AfterDelete(tx *gorm.DB) error }
func WithSoftDeleteCascade(tx *gorm.DB, cascade func(tx *gorm.DB) error) error

From summercms.go/lagoon/migrations.go (existing, this plan edits its Migrate func only):

func migrator(gdb *gorm.DB, pluginID string, migrations []*gormigrate.Migration) (*gormigrate.Gormigrate, error)
func Migrate(gdb *gorm.DB, plugins []party.Plugin) error

From fonoteka.go/plugins/golem15/fonoteka/models/album.go, models/collection.go (Plan 05-02, widened):

type Album struct { ID uint; ... }       // gains a MorphName() method and photo relation this plan
type Collection struct { ID uint; ... }  // gains MorphName(), photos (ordered), image (single) this plan
Task 1 (summercms.go): lagoon/attach core — File model, system_files migration wired before every plugin set, blob bucket, Thumb() naming with lazy imaging resize summercms.go/go.mod, summercms.go/go.sum, summercms.go/lagoon/attach/file.go, attach/file_test.go, attach/migrations.go, attach/migrations_test.go, attach/bucket.go, attach/thumb.go, attach/thumb_test.go, summercms.go/lagoon/migrations.go, migrations_test.go, fonoteka.go/config/storage.yaml /media/nvme/dev/golem15/fonoteka/vendor/winter/storm/src/Database/Attach/File.php (lines ~440-1050: getThumbFilename, getPartitionDirectory, getStorageDirectory, delete behavior) /media/nvme/dev/golem15/fonoteka/modules/system/database/migrations/2013_10_01_000002_Db_System_Files.php /media/nvme/dev/golem15/fonoteka/config/cms.php (lines ~300-335, storage.uploads config shape) summercms.go/lagoon/migrations.go (migrator() helper, Migrate() loop to modify) summercms.go/lagoon/connection.go (Publish/one-shared-handle discipline) summercms.go/backpack/services.go (Registry.Publish/Lookup) .planning/phases/05-data-layer-full-fidelity/05-RESEARCH.md ("system_files verified column set", "Winter File thumb filename + partition rule" code example, "disintegration/imaging thumbnail generation" code example, D-14..D-18 decisions) - `Thumb(42, 200, 200, "0", "0", "crop", "jpg")`-shaped naming call (or the equivalent method signature on `File`) returns exactly `"thumb_42_200_200_0_0_crop.jpg"`. - Partition directory for `disk_name="abc123xyz.jpg"` is exactly `"abc/123/xyz/"`. - Calling `File.Thumb(gdb, bucket, 200, 200, "crop")` twice on the same file only invokes the resize path once (second call finds the existing thumb blob key and returns its URL without re-resizing) — verified against a `memblob` bucket with a resize-call counter. - `lagoon.Migrate` run against a fresh database creates `system_files` before any plugin's own tables (verified by asserting the history/table exists even when the plugin list is empty). Add dependencies: `cd summercms.go && go get gocloud.dev/blob@v0.46.0 && go get github.com/disintegration/imaging@v1.6.2` (both are `[OK]`/Approved in RESEARCH.md's Package Legitimacy Audit — no blocking checkpoint needed).
Create `lagoon/attach/file.go` (package `attach`) with `type File struct { ID uint; DiskName string; FileName string; FileSize int64; ContentType string; Title *string; Description *string; Field string; AttachmentID string; AttachmentType string; IsPublic bool; SortOrder int; Metadata *string; CreatedAt time.Time; UpdatedAt time.Time }` and `func (File) TableName() string { return "system_files" }` — column names exactly as RESEARCH.md's verified `system_files` set, noting `AttachmentID string` (Winter's morph FK is a string, never an integer, per the verified live-schema note). Declare `type Owner interface { MorphName() string }` — any Fonoteka model implements this to keep the PHP class string (e.g. `"Golem15\\Fonoteka\\Models\\Album"`) in `AttachmentType` on save. `File` self-registers via `attach.Register` — reuse the exact `models.Register`/`All` registry shape from Plan 05-01 but scoped to this package (`var all []any; func Register(...); func All() []any`) so `lagoon.Migrate`/schema tooling can see it without depending on any single plugin's registry.

Create `lagoon/attach/migrations.go` with `var Migrations = []*gormigrate.Migration{{ID: "<date>0001_create_system_files", Migrate: func(tx *gorm.DB) error { /* raw CREATE TABLE system_files per RESEARCH.md's verified column set, plus an index on (attachment_type, attachment_id, field) */ return nil }, Rollback: func(tx *gorm.DB) error { return tx.Exec("DROP TABLE IF EXISTS system_files").Error }}}`.

Edit `lagoon/migrations.go`'s existing `Migrate(gdb *gorm.DB, plugins []party.Plugin) error`: as its very first statement (before the `for _, p := range plugins` loop), call `m, err := migrator(gdb, "summercms.attach", attach.Migrations); if err != nil { return err }; if err := m.Migrate(); err != nil { return fmt.Errorf("lagoon: migrate system_files: %w", err) }` (reusing the existing unexported `migrator()` helper with a synthetic plugin id — no plugin needs to know this ran). This is the "runs before every plugin set" wiring (D-14); it is the only line this task adds to a file another plan might also touch — no other Phase 5 plan edits `lagoon/migrations.go`, so this is collision-free.

Create `lagoon/attach/bucket.go` with `func OpenBucket(ctx context.Context, cfg *compass.Config) (*blob.Bucket, error)` reading `storage.uploads.bucket_url` (a `fileblob://` or `mem://` URL per `gocloud.dev/blob`'s `blob.OpenBucket` convention) and `storage.uploads.public_path_prefix` from config, failing loudly if `bucket_url` is empty (same fail-boot discipline as `app.key`/`jwt.secret`); publish the opened bucket once via `app.Publish(bucket)` (mirror `lagoon.Publish`'s one-shared-handle discipline) — add `fonoteka.go/config/storage.yaml` with `uploads:\n  bucket_url: "file://./storage/app/uploads"\n  public_path_prefix: "/storage/uploads"` mirroring `cms.php`'s `storage.uploads` shape (D-16).

Create `lagoon/attach/thumb.go` implementing the two pure-string functions verbatim from RESEARCH.md's verified Winter source: `func ThumbFilename(id uint, w, h int, offsetX, offsetY int, mode, ext string) string` (`"thumb_%d_%d_%d_%d_%d_%s.%s"`) and `func PartitionDirectory(diskName string) string` (first 9 chars of `diskName` split into 3 groups of 3, joined by `/`, trailing `/`). Implement `func (f *File) Thumb(ctx context.Context, bucket *blob.Bucket, w, h int, mode string) (string, error)` that computes the thumb key via the two functions above, checks `bucket.Exists(ctx, thumbKey)` first (lazy generation — do not resize if the thumb blob already exists, satisfying the "reused at cutover" requirement and the "only resize once" behavior test), and only on a cache miss reads the original via `bucket.NewReader`, decodes with `image.Decode`, resizes with `imaging.Fill`/`imaging.Resize`/`imaging.Fit` per the mode-to-function mapping in RESEARCH.md's code example (`crop`->`Fill`, `exact`->`Resize`, default `auto`->`Fit`, all with `imaging.Lanczos`), writes the result via `bucket.NewWriter`, and returns the computed public URL (`public_path_prefix` + partition + filename).
cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./lagoon/... && go test ./lagoon/... -run 'TestThumbFilename|TestPartitionDirectory|TestFileLifecycle|TestMigrateRunsSystemFilesFirst' -short - `ThumbFilename(42, 200, 200, 0, 0, "crop", "jpg")` returns exactly `"thumb_42_200_200_0_0_crop.jpg"`. - `PartitionDirectory("abc123xyz.jpg")` returns exactly `"abc/123/xyz/"`. - `File.Thumb` called twice against a `memblob` bucket resizes exactly once (assert via a counting `imaging` wrapper or by asserting the second call's `bucket.NewReader` on the original is never opened). - A test asserts `system_files` exists immediately after `lagoon.Migrate(gdb, nil)` (empty plugin list) — proving the framework migration runs independent of any plugin. - `go.mod` pins `gocloud.dev v0.46.0` and `github.com/disintegration/imaging v1.6.2`. `lagoon/attach` exists with `File`, the `system_files` migration wired ahead of every plugin set, blob bucket wiring, and `Thumb()` matching Winter's exact naming with lazy, cached resize via `disintegration/imaging`. Task 2 (summercms.go): delete lifecycle (soft-delete keeps blobs, force-delete removes after commit) and a static file handler for public attachments summercms.go/lagoon/attach/file.go, summercms.go/lagoon/attach/file_test.go, summercms.go/lagoon/attach/static.go, summercms.go/lagoon/attach/static_test.go /media/nvme/dev/golem15/fonoteka/vendor/winter/storm/src/Database/Attach/File.php (delete method — confirm the exact soft-delete-keeps-row-and-blob vs. force-delete-removes behavior) summercms.go/lagoon/lifecycle.go (Plan 05-01, HasAfterDelete, WithSoftDeleteCascade) .planning/phases/05-data-layer-full-fidelity/05-CONTEXT.md (D-16, D-19) - Soft-deleting an owner (e.g. `tx.Delete(&Album{...})` where `Album` has `gorm.DeletedAt`) leaves its `system_files` rows and blob bytes untouched. - Force-deleting an owner (`tx.Unscoped().Delete(...)`) removes the `system_files` row inside the same transaction as the owner's delete, and the original blob plus any generated thumbs are removed from the bucket only after the transaction commits (never inside it — a rollback must not have already deleted bytes it can't get back). - `attach.StaticHandler(bucket, prefix)` serves a stored blob's bytes with the correct `Content-Type` at `prefix//` via `httptest.NewServer`, and 404s for a missing key; unvalidated path segments never reach `bucket.NewReader`. Add `func DeleteForOwner(tx *gorm.DB, owner Owner, ownerID string, afterCommit func(blobKeys []string) error) error` to `file.go`: inside `tx`, `SELECT disk_name FROM system_files WHERE attachment_type = ? AND attachment_id = ?` for the rows about to be removed, `DELETE` those rows in the same statement/transaction, then register `afterCommit` to run once `tx.Commit()` succeeds — GORM does not have a native "after commit" hook, so implement this by having the caller (a model's `AfterDelete(tx *gorm.DB) error` hook, wired the same way Plan 05-02's `ArtistResolver` callback was wired via `classes.RegisterHook`) collect blob keys during the transaction and issue the actual `bucket.Delete` calls immediately after the top-level `.Unscoped().Delete(...)` call returns without error in the caller's own code — document this two-phase contract precisely in a doc comment on `DeleteForOwner`, since it is a real GORM limitation (no post-commit hook), not an oversight. Only wire this contract into the framework helper here; the first real caller (Album/Collection force-delete) is exercised by this plan's Task 3 smoke test, not a production route (none exists yet).
In `static.go` implement `func StaticHandler(bucket *blob.Bucket, prefix string) http.Handler` (D-16): serve GET `prefix/<partition>/<disk_name>` by reconstructing the blob key from `PartitionDirectory(disk_name)+disk_name` (the Winter 3x3 partition rule Task 1 already shipped) and reading that exact key via `bucket.NewReader`. Set `Content-Type` from the blob's attributes if present, otherwise from a documented default such as `application/octet-stream`. Exact-match keys only: reject `..`, extra slashes, empty segments, and any path that is not exactly the 3-group partition plus a single `disk_name` file component; never pass unvalidated request path segments to `bucket.NewReader` (T-05-13). A missing key returns 404. Do NOT register this handler on any plugin route (D-16 / this phase's HTTP-route boundary) — `static_test.go`'s `TestStaticHandler` drives it only through `httptest.NewServer`.
cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./lagoon/... && go test ./lagoon/attach/... -run 'TestFileLifecycle|TestStaticHandler' - `TestFileLifecycle` proves soft-delete keeps the row+blob and force-delete removes the row in-tx with blob removal deferred to after commit (assert the blob still exists immediately inside a test transaction that has not yet committed, then confirm it is gone after commit). - `attach.StaticHandler(bucket, prefix)` serves a stored blob's bytes with the correct `Content-Type` at `prefix//` via `httptest.NewServer`, and 404s for a missing key — no route registration in any plugin, per this phase's HTTP-route boundary. Delete lifecycle matches Winter's soft-delete/force-delete split with post-commit blob removal; a static handler exists and is tested directly, not yet wired into any route table. Task 3 (fonoteka.go): Album/Collection attachment wiring — MorphName, ordered photos, Collection's single image fonoteka.go/plugins/golem15/fonoteka/models/album.go, fonoteka.go/plugins/golem15/fonoteka/models/collection.go /media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/models/Album.php (attachMany photos declaration) /media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/models/Collection.php (lines ~91-107: attachMany photos ordered, attachOne image) /media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/traits/SerializesFonoteka.php (lines ~190-215: photo payload shape, getThumb(200,200,['mode'=>'crop']), relativeMediaUrl — port only what this plan's own smoke test needs) summercms.go/lagoon/attach/file.go (Owner interface, this plan's Task 1) Add `func (Album) MorphName() string { return "Golem15\\Fonoteka\\Models\\Album" }` and `func (Collection) MorphName() string { return "Golem15\\Fonoteka\\Models\\Collection" }` to the respective model files — the literal PHP class strings (verify exact casing/namespace against the PHP files' own `namespace`/class declarations), satisfying `attach.Owner` so cutover-copied `system_files` rows keep matching against these Go models unchanged. Document in a comment that `Album`'s photos are `attachMany` (ordered by `sort_order`, queried via `system_files WHERE attachment_type = Album.MorphName() AND attachment_id = ? AND field = 'photos' ORDER BY sort_order`) and `Collection` has both an ordered `photos` `attachMany` and a single `image` `attachOne` (`field = 'image'`, at most one row) — these are query helpers on the `classes/` write-service layer (not a new file this task adds; note the follow-up call site for Plan 05-06's smoke test), not new relation-tag machinery, since `system_files` is framework-owned and polymorphic rather than a GORM-native relation. cd /media/nvme/dev/golem15/summercms.io/summercms/fonoteka.go && go build ./... && go vet ./... - `grep -n "MorphName" fonoteka.go/plugins/golem15/fonoteka/models/album.go fonoteka.go/plugins/golem15/fonoteka/models/collection.go` shows both methods present with the exact PHP class-string literals. - `go vet` and `go build` are clean; no new relation tag or migration is introduced (attachments stay entirely framework-owned via `system_files`, per D-14). Album and Collection satisfy `attach.Owner` with the exact PHP class strings; Collection's ordered photos and single image are documented query shapes ready for Plan 05-06's smoke test.

<threat_model>

Trust Boundaries

Boundary Description
stored blob path -> static handler disk_name/partition path must never allow traversal outside the configured bucket root
force-delete -> blob removal timing a rolled-back transaction must never have already deleted bytes it cannot restore

STRIDE Threat Register

Threat ID Category Component Disposition Mitigation Plan
T-05-13 Tampering (path traversal) attach.StaticHandler key construction mitigate Keys are built only from disk_name values already persisted by system_files rows (server-generated, never taken from the request path beyond an exact-match lookup); the handler does not accept caller-supplied directory segments that reach bucket.NewReader unvalidated
T-05-14 Tampering (data loss) Force-delete blob removal ordering mitigate DeleteForOwner's documented two-phase contract removes DB rows inside the transaction and blobs only after commit succeeds, so a rollback never leaves orphaned-but-unrecoverable bytes deleted early
T-05-15 Tampering (supply chain) gocloud.dev/blob, disintegration/imaging accept Both [OK] in RESEARCH.md's Package Legitimacy Audit; no blocking human-verify checkpoint required
T-05-16 Information Disclosure attach.StaticHandler serving is_public=false rows accept This phase's StaticHandler is not yet wired into any route (HTTP-route boundary of this phase); the is_public gate is Phase 6/12's job when the handler is actually mounted — flagged here so it is not forgotten at wiring time
</threat_model>
`cd summercms.go && go vet ./lagoon/... && go test ./lagoon/attach/... -short` then the full non-short suite (testcontainers Postgres + memblob); `cd ../fonoteka.go && go build ./... && go vet ./...`.

<success_criteria>

  • system_files migrates before every plugin's own set, verified with an empty plugin list.
  • Thumb() matches Winter's exact filename/partition rules and resizes lazily, once, via disintegration/imaging.
  • Soft-delete/force-delete lifecycle matches D-19 exactly, including deferred post-commit blob removal.
  • Album/Collection implement attach.Owner with the correct PHP class strings. </success_criteria>
Create `.planning/phases/05-data-layer-full-fidelity/05-04-SUMMARY.md` when done