Files
summercms/.planning/phases/05-data-layer-full-fidelity/05-04-PLAN.md
2026-09-18 17:31:53 +02:00

230 lines
22 KiB
Markdown

---
phase: 05-data-layer-full-fidelity
plan: 04
type: execute
wave: 3
depends_on: ["05-02"]
files_modified:
- summercms.go/go.mod
- summercms.go/go.sum
- summercms.go/lagoon/attach/file.go
- summercms.go/lagoon/attach/file_test.go
- summercms.go/lagoon/attach/migrations.go
- summercms.go/lagoon/attach/migrations_test.go
- summercms.go/lagoon/attach/bucket.go
- summercms.go/lagoon/attach/thumb.go
- summercms.go/lagoon/attach/thumb_test.go
- summercms.go/lagoon/attach/static.go
- summercms.go/lagoon/attach/static_test.go
- summercms.go/lagoon/migrations.go
- summercms.go/lagoon/migrations_test.go
- fonoteka.go/config/storage.yaml
- fonoteka.go/plugins/golem15/fonoteka/models/album.go
- fonoteka.go/plugins/golem15/fonoteka/models/collection.go
autonomous: true
requirements: [DATA-08, DATA-09]
must_haves:
truths:
- "The framework-owned system_files table (Winter's exact shape and column names) migrates before every plugin's own migration set runs, via lagoon.Migrate itself, not an app-level wiring call (D-14)"
- "Thumb(w,h,mode) reproduces Winter's exact filename (thumb_<id>_<w>_<h>_0_0_<mode>.<ext>) and partition-directory rule, generating lazily through gocloud.dev/blob on first call via disintegration/imaging (D-15, D-16, D-17, D-18)"
- "Files are stored/deleted through gocloud.dev/blob (fileblob in dev/prod rooted at storage/app/uploads, memblob in unit tests), never direct os.WriteFile/ReadFile (D-15, D-16)"
- "Soft-deleting an attachment's owner keeps system_files rows and blobs; force-deleting removes the row inside the delete transaction and the blob (plus thumbs) only after commit (D-19)"
- "Album and Collection carry per-model morph names that keep the PHP class string in attachment_type, and Collection's photos are ordered while its image is a single attachOne (Collection.php ~91-107)"
artifacts:
- path: summercms.go/lagoon/attach/file.go
provides: "File model (system_files) + Owner interface (MorphName)"
- path: summercms.go/lagoon/attach/migrations.go
provides: "create_system_files migration, run before every plugin set"
- path: summercms.go/lagoon/attach/thumb.go
provides: "Thumb(w,h,mode) naming + lazy disintegration/imaging resize"
- path: summercms.go/lagoon/attach/static.go
provides: "StaticHandler(bucket, prefix) http.Handler for public files (not yet route-wired — Phase 6 territory)"
key_links:
- from: summercms.go/lagoon/migrations.go
to: summercms.go/lagoon/attach/migrations.go
via: "Migrate() runs attach's migration set first, before iterating plugins"
pattern: "attach\\.Migrate\\(|attach\\.Migrations"
- from: fonoteka.go/plugins/golem15/fonoteka/models/album.go
to: summercms.go/lagoon/attach/file.go
via: "Album implements attach.Owner via MorphName() returning the PHP class string"
pattern: "func \\(Album\\) MorphName\\(\\) string"
---
<objective>
Ship the framework-owned `system_files` attachment table and `File` model, the `gocloud.dev/blob` bucket wiring (fileblob/memblob), Winter-exact `Thumb()` naming with lazy `disintegration/imaging` resize, the delete-lifecycle rule (soft-delete keeps blobs, force-delete removes them after commit), and wire `Album`'s ordered photos and `Collection`'s photos/image onto it — the full D-15 storage scope the user chose over the smaller table-only cut.
Purpose: this is the phase's largest single new subsystem (first blob-storage code in the repo) and the plan most likely to blow context if under-scoped — it ships storage end to end (metadata + bytes + thumbnails + lifecycle) but stops at the HTTP upload endpoint, which stays out of scope until Phase 12 per the phase boundary.
Output: `lagoon/attach` package (File, migrations, bucket, thumb, static handler); `disintegration/imaging` and `gocloud.dev/blob` added to `summercms.go/go.mod`; `Album`/`Collection` attachment wiring in `fonoteka.go`.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/phases/05-data-layer-full-fidelity/05-CONTEXT.md
@.planning/phases/05-data-layer-full-fidelity/05-RESEARCH.md
@.planning/phases/05-data-layer-full-fidelity/05-02-SUMMARY.md
</context>
<interfaces>
<!-- Contracts from Plans 05-01/05-02 this plan's code consumes -->
From summercms.go/lagoon/lifecycle.go (Plan 05-01):
```go
type HasAfterDelete interface { AfterDelete(tx *gorm.DB) error }
func WithSoftDeleteCascade(tx *gorm.DB, cascade func(tx *gorm.DB) error) error
```
From summercms.go/lagoon/migrations.go (existing, this plan edits its `Migrate` func only):
```go
func migrator(gdb *gorm.DB, pluginID string, migrations []*gormigrate.Migration) (*gormigrate.Gormigrate, error)
func Migrate(gdb *gorm.DB, plugins []party.Plugin) error
```
From fonoteka.go/plugins/golem15/fonoteka/models/album.go, models/collection.go (Plan 05-02, widened):
```go
type Album struct { ID uint; ... } // gains a MorphName() method and photo relation this plan
type Collection struct { ID uint; ... } // gains MorphName(), photos (ordered), image (single) this plan
```
</interfaces>
<tasks>
<task type="auto" tdd="true">
<name>Task 1 (summercms.go): lagoon/attach core — File model, system_files migration wired before every plugin set, blob bucket, Thumb() naming with lazy imaging resize</name>
<files>
summercms.go/go.mod, summercms.go/go.sum,
summercms.go/lagoon/attach/file.go, attach/file_test.go, attach/migrations.go, attach/migrations_test.go, attach/bucket.go, attach/thumb.go, attach/thumb_test.go,
summercms.go/lagoon/migrations.go, migrations_test.go,
fonoteka.go/config/storage.yaml
</files>
<read_first>
/media/nvme/dev/golem15/fonoteka/vendor/winter/storm/src/Database/Attach/File.php (lines ~440-1050: getThumbFilename, getPartitionDirectory, getStorageDirectory, delete behavior)
/media/nvme/dev/golem15/fonoteka/modules/system/database/migrations/2013_10_01_000002_Db_System_Files.php
/media/nvme/dev/golem15/fonoteka/config/cms.php (lines ~300-335, storage.uploads config shape)
summercms.go/lagoon/migrations.go (migrator() helper, Migrate() loop to modify)
summercms.go/lagoon/connection.go (Publish/one-shared-handle discipline)
summercms.go/backpack/services.go (Registry.Publish/Lookup)
.planning/phases/05-data-layer-full-fidelity/05-RESEARCH.md ("system_files verified column set", "Winter File thumb filename + partition rule" code example, "disintegration/imaging thumbnail generation" code example, D-14..D-18 decisions)
</read_first>
<behavior>
- `Thumb(42, 200, 200, "0", "0", "crop", "jpg")`-shaped naming call (or the equivalent method signature on `File`) returns exactly `"thumb_42_200_200_0_0_crop.jpg"`.
- Partition directory for `disk_name="abc123xyz.jpg"` is exactly `"abc/123/xyz/"`.
- Calling `File.Thumb(gdb, bucket, 200, 200, "crop")` twice on the same file only invokes the resize path once (second call finds the existing thumb blob key and returns its URL without re-resizing) — verified against a `memblob` bucket with a resize-call counter.
- `lagoon.Migrate` run against a fresh database creates `system_files` before any plugin's own tables (verified by asserting the history/table exists even when the plugin list is empty).
</behavior>
<action>
Add dependencies: `cd summercms.go && go get gocloud.dev/blob@v0.46.0 && go get github.com/disintegration/imaging@v1.6.2` (both are `[OK]`/Approved in RESEARCH.md's Package Legitimacy Audit — no blocking checkpoint needed).
Create `lagoon/attach/file.go` (package `attach`) with `type File struct { ID uint; DiskName string; FileName string; FileSize int64; ContentType string; Title *string; Description *string; Field string; AttachmentID string; AttachmentType string; IsPublic bool; SortOrder int; Metadata *string; CreatedAt time.Time; UpdatedAt time.Time }` and `func (File) TableName() string { return "system_files" }` — column names exactly as RESEARCH.md's verified `system_files` set, noting `AttachmentID string` (Winter's morph FK is a string, never an integer, per the verified live-schema note). Declare `type Owner interface { MorphName() string }` — any Fonoteka model implements this to keep the PHP class string (e.g. `"Golem15\\Fonoteka\\Models\\Album"`) in `AttachmentType` on save. `File` self-registers via `attach.Register` — reuse the exact `models.Register`/`All` registry shape from Plan 05-01 but scoped to this package (`var all []any; func Register(...); func All() []any`) so `lagoon.Migrate`/schema tooling can see it without depending on any single plugin's registry.
Create `lagoon/attach/migrations.go` with `var Migrations = []*gormigrate.Migration{{ID: "<date>0001_create_system_files", Migrate: func(tx *gorm.DB) error { /* raw CREATE TABLE system_files per RESEARCH.md's verified column set, plus an index on (attachment_type, attachment_id, field) */ return nil }, Rollback: func(tx *gorm.DB) error { return tx.Exec("DROP TABLE IF EXISTS system_files").Error }}}`.
Edit `lagoon/migrations.go`'s existing `Migrate(gdb *gorm.DB, plugins []party.Plugin) error`: as its very first statement (before the `for _, p := range plugins` loop), call `m, err := migrator(gdb, "summercms.attach", attach.Migrations); if err != nil { return err }; if err := m.Migrate(); err != nil { return fmt.Errorf("lagoon: migrate system_files: %w", err) }` (reusing the existing unexported `migrator()` helper with a synthetic plugin id — no plugin needs to know this ran). This is the "runs before every plugin set" wiring (D-14); it is the only line this task adds to a file another plan might also touch — no other Phase 5 plan edits `lagoon/migrations.go`, so this is collision-free.
Create `lagoon/attach/bucket.go` with `func OpenBucket(ctx context.Context, cfg *compass.Config) (*blob.Bucket, error)` reading `storage.uploads.bucket_url` (a `fileblob://` or `mem://` URL per `gocloud.dev/blob`'s `blob.OpenBucket` convention) and `storage.uploads.public_path_prefix` from config, failing loudly if `bucket_url` is empty (same fail-boot discipline as `app.key`/`jwt.secret`); publish the opened bucket once via `app.Publish(bucket)` (mirror `lagoon.Publish`'s one-shared-handle discipline) — add `fonoteka.go/config/storage.yaml` with `uploads:\n bucket_url: "file://./storage/app/uploads"\n public_path_prefix: "/storage/uploads"` mirroring `cms.php`'s `storage.uploads` shape (D-16).
Create `lagoon/attach/thumb.go` implementing the two pure-string functions verbatim from RESEARCH.md's verified Winter source: `func ThumbFilename(id uint, w, h int, offsetX, offsetY int, mode, ext string) string` (`"thumb_%d_%d_%d_%d_%d_%s.%s"`) and `func PartitionDirectory(diskName string) string` (first 9 chars of `diskName` split into 3 groups of 3, joined by `/`, trailing `/`). Implement `func (f *File) Thumb(ctx context.Context, bucket *blob.Bucket, w, h int, mode string) (string, error)` that computes the thumb key via the two functions above, checks `bucket.Exists(ctx, thumbKey)` first (lazy generation — do not resize if the thumb blob already exists, satisfying the "reused at cutover" requirement and the "only resize once" behavior test), and only on a cache miss reads the original via `bucket.NewReader`, decodes with `image.Decode`, resizes with `imaging.Fill`/`imaging.Resize`/`imaging.Fit` per the mode-to-function mapping in RESEARCH.md's code example (`crop`->`Fill`, `exact`->`Resize`, default `auto`->`Fit`, all with `imaging.Lanczos`), writes the result via `bucket.NewWriter`, and returns the computed public URL (`public_path_prefix` + partition + filename).
</action>
<verify>
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./lagoon/... && go test ./lagoon/... -run 'TestThumbFilename|TestPartitionDirectory|TestFileLifecycle|TestMigrateRunsSystemFilesFirst' -short</automated>
</verify>
<acceptance_criteria>
- `ThumbFilename(42, 200, 200, 0, 0, "crop", "jpg")` returns exactly `"thumb_42_200_200_0_0_crop.jpg"`.
- `PartitionDirectory("abc123xyz.jpg")` returns exactly `"abc/123/xyz/"`.
- `File.Thumb` called twice against a `memblob` bucket resizes exactly once (assert via a counting `imaging` wrapper or by asserting the second call's `bucket.NewReader` on the original is never opened).
- A test asserts `system_files` exists immediately after `lagoon.Migrate(gdb, nil)` (empty plugin list) — proving the framework migration runs independent of any plugin.
- `go.mod` pins `gocloud.dev v0.46.0` and `github.com/disintegration/imaging v1.6.2`.
</acceptance_criteria>
<done>`lagoon/attach` exists with `File`, the `system_files` migration wired ahead of every plugin set, blob bucket wiring, and `Thumb()` matching Winter's exact naming with lazy, cached resize via `disintegration/imaging`.</done>
</task>
<task type="auto">
<name>Task 2 (summercms.go): delete lifecycle (soft-delete keeps blobs, force-delete removes after commit) and a static file handler for public attachments</name>
<files>summercms.go/lagoon/attach/file.go, summercms.go/lagoon/attach/file_test.go, summercms.go/lagoon/attach/static.go, summercms.go/lagoon/attach/static_test.go</files>
<read_first>
/media/nvme/dev/golem15/fonoteka/vendor/winter/storm/src/Database/Attach/File.php (delete method — confirm the exact soft-delete-keeps-row-and-blob vs. force-delete-removes behavior)
summercms.go/lagoon/lifecycle.go (Plan 05-01, HasAfterDelete, WithSoftDeleteCascade)
.planning/phases/05-data-layer-full-fidelity/05-CONTEXT.md (D-16, D-19)
</read_first>
<behavior>
- Soft-deleting an owner (e.g. `tx.Delete(&Album{...})` where `Album` has `gorm.DeletedAt`) leaves its `system_files` rows and blob bytes untouched.
- Force-deleting an owner (`tx.Unscoped().Delete(...)`) removes the `system_files` row inside the same transaction as the owner's delete, and the original blob plus any generated thumbs are removed from the bucket only after the transaction commits (never inside it — a rollback must not have already deleted bytes it can't get back).
- `attach.StaticHandler(bucket, prefix)` serves a stored blob's bytes with the correct `Content-Type` at `prefix/<partition>/<disk_name>` via `httptest.NewServer`, and 404s for a missing key; unvalidated path segments never reach `bucket.NewReader`.
</behavior>
<action>
Add `func DeleteForOwner(tx *gorm.DB, owner Owner, ownerID string, afterCommit func(blobKeys []string) error) error` to `file.go`: inside `tx`, `SELECT disk_name FROM system_files WHERE attachment_type = ? AND attachment_id = ?` for the rows about to be removed, `DELETE` those rows in the same statement/transaction, then register `afterCommit` to run once `tx.Commit()` succeeds — GORM does not have a native "after commit" hook, so implement this by having the caller (a model's `AfterDelete(tx *gorm.DB) error` hook, wired the same way Plan 05-02's `ArtistResolver` callback was wired via `classes.RegisterHook`) collect blob keys during the transaction and issue the actual `bucket.Delete` calls immediately after the top-level `.Unscoped().Delete(...)` call returns without error in the caller's own code — document this two-phase contract precisely in a doc comment on `DeleteForOwner`, since it is a real GORM limitation (no post-commit hook), not an oversight. Only wire this contract into the framework helper here; the first real caller (Album/Collection force-delete) is exercised by this plan's Task 3 smoke test, not a production route (none exists yet).
In `static.go` implement `func StaticHandler(bucket *blob.Bucket, prefix string) http.Handler` (D-16): serve GET `prefix/<partition>/<disk_name>` by reconstructing the blob key from `PartitionDirectory(disk_name)+disk_name` (the Winter 3x3 partition rule Task 1 already shipped) and reading that exact key via `bucket.NewReader`. Set `Content-Type` from the blob's attributes if present, otherwise from a documented default such as `application/octet-stream`. Exact-match keys only: reject `..`, extra slashes, empty segments, and any path that is not exactly the 3-group partition plus a single `disk_name` file component; never pass unvalidated request path segments to `bucket.NewReader` (T-05-13). A missing key returns 404. Do NOT register this handler on any plugin route (D-16 / this phase's HTTP-route boundary) — `static_test.go`'s `TestStaticHandler` drives it only through `httptest.NewServer`.
</action>
<verify>
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./lagoon/... && go test ./lagoon/attach/... -run 'TestFileLifecycle|TestStaticHandler'</automated>
</verify>
<acceptance_criteria>
- `TestFileLifecycle` proves soft-delete keeps the row+blob and force-delete removes the row in-tx with blob removal deferred to after commit (assert the blob still exists immediately inside a test transaction that has not yet committed, then confirm it is gone after commit).
- `attach.StaticHandler(bucket, prefix)` serves a stored blob's bytes with the correct `Content-Type` at `prefix/<partition>/<disk_name>` via `httptest.NewServer`, and 404s for a missing key — no route registration in any plugin, per this phase's HTTP-route boundary.
</acceptance_criteria>
<done>Delete lifecycle matches Winter's soft-delete/force-delete split with post-commit blob removal; a static handler exists and is tested directly, not yet wired into any route table.</done>
</task>
<task type="auto">
<name>Task 3 (fonoteka.go): Album/Collection attachment wiring — MorphName, ordered photos, Collection's single image</name>
<files>fonoteka.go/plugins/golem15/fonoteka/models/album.go, fonoteka.go/plugins/golem15/fonoteka/models/collection.go</files>
<read_first>
/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/models/Album.php (attachMany photos declaration)
/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/models/Collection.php (lines ~91-107: attachMany photos ordered, attachOne image)
/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/traits/SerializesFonoteka.php (lines ~190-215: photo payload shape, getThumb(200,200,['mode'=>'crop']), relativeMediaUrl — port only what this plan's own smoke test needs)
summercms.go/lagoon/attach/file.go (Owner interface, this plan's Task 1)
</read_first>
<action>
Add `func (Album) MorphName() string { return "Golem15\\Fonoteka\\Models\\Album" }` and `func (Collection) MorphName() string { return "Golem15\\Fonoteka\\Models\\Collection" }` to the respective model files — the literal PHP class strings (verify exact casing/namespace against the PHP files' own `namespace`/class declarations), satisfying `attach.Owner` so cutover-copied `system_files` rows keep matching against these Go models unchanged. Document in a comment that `Album`'s photos are `attachMany` (ordered by `sort_order`, queried via `system_files WHERE attachment_type = Album.MorphName() AND attachment_id = ? AND field = 'photos' ORDER BY sort_order`) and `Collection` has both an ordered `photos` `attachMany` and a single `image` `attachOne` (`field = 'image'`, at most one row) — these are query helpers on the `classes/` write-service layer (not a new file this task adds; note the follow-up call site for Plan 05-06's smoke test), not new relation-tag machinery, since `system_files` is framework-owned and polymorphic rather than a GORM-native relation.
</action>
<verify>
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/fonoteka.go && go build ./... && go vet ./...</automated>
</verify>
<acceptance_criteria>
- `grep -n "MorphName" fonoteka.go/plugins/golem15/fonoteka/models/album.go fonoteka.go/plugins/golem15/fonoteka/models/collection.go` shows both methods present with the exact PHP class-string literals.
- `go vet` and `go build` are clean; no new relation tag or migration is introduced (attachments stay entirely framework-owned via `system_files`, per D-14).
</acceptance_criteria>
<done>Album and Collection satisfy `attach.Owner` with the exact PHP class strings; Collection's ordered photos and single image are documented query shapes ready for Plan 05-06's smoke test.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|--------------|
| stored blob path -> static handler | `disk_name`/partition path must never allow traversal outside the configured bucket root |
| force-delete -> blob removal timing | a rolled-back transaction must never have already deleted bytes it cannot restore |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-05-13 | Tampering (path traversal) | `attach.StaticHandler` key construction | mitigate | Keys are built only from `disk_name` values already persisted by `system_files` rows (server-generated, never taken from the request path beyond an exact-match lookup); the handler does not accept caller-supplied directory segments that reach `bucket.NewReader` unvalidated |
| T-05-14 | Tampering (data loss) | Force-delete blob removal ordering | mitigate | `DeleteForOwner`'s documented two-phase contract removes DB rows inside the transaction and blobs only after commit succeeds, so a rollback never leaves orphaned-but-unrecoverable bytes deleted early |
| T-05-15 | Tampering (supply chain) | `gocloud.dev/blob`, `disintegration/imaging` | accept | Both `[OK]` in RESEARCH.md's Package Legitimacy Audit; no blocking human-verify checkpoint required |
| T-05-16 | Information Disclosure | `attach.StaticHandler` serving `is_public=false` rows | accept | This phase's `StaticHandler` is not yet wired into any route (HTTP-route boundary of this phase); the `is_public` gate is Phase 6/12's job when the handler is actually mounted — flagged here so it is not forgotten at wiring time |
</threat_model>
<verification>
`cd summercms.go && go vet ./lagoon/... && go test ./lagoon/attach/... -short` then the full non-short suite (testcontainers Postgres + memblob); `cd ../fonoteka.go && go build ./... && go vet ./...`.
</verification>
<success_criteria>
- `system_files` migrates before every plugin's own set, verified with an empty plugin list.
- `Thumb()` matches Winter's exact filename/partition rules and resizes lazily, once, via `disintegration/imaging`.
- Soft-delete/force-delete lifecycle matches D-19 exactly, including deferred post-commit blob removal.
- `Album`/`Collection` implement `attach.Owner` with the correct PHP class strings.
</success_criteria>
<output>
Create `.planning/phases/05-data-layer-full-fidelity/05-04-SUMMARY.md` when done
</output>