624 lines
81 KiB
Markdown
624 lines
81 KiB
Markdown
# Phase 5: Data layer full fidelity - Research
|
||
|
||
**Researched:** 2026-09-18
|
||
**Domain:** GORM/Postgres model porting from a WinterCMS/Laravel source (25 models, mass-assignment/encryption security invariants, polymorphic attachments)
|
||
**Confidence:** HIGH
|
||
|
||
<user_constraints>
|
||
## User Constraints (from CONTEXT.md)
|
||
|
||
### Locked Decisions
|
||
|
||
**Migration granularity**
|
||
- D-01: Go migrations stay squashed per final-state table (P3 D-16 stands). One create migration per table, or per small tightly-coupled table group (e.g. OAuth tables, CSV import tables, the two Discogs credential tables), in FK order, each carrying a comment naming the PHP update files it folds. The PHP rename/flatten history (areas, items, locations, tags, item_categories) is not replayed. Success criterion 1 is read as: every Go migration runs up and down individually, and the final schema matches PHP's. The roadmap's "27" is not a target; the planner should fix the wording in ROADMAP.md/REQUIREMENTS.md DATA-09 when the phase is planned.
|
||
- D-02: Schema equality is proven by an automated diff, not by reading. A normalized snapshot of the PHP app's final schema on Postgres (tables, columns, types, nullability, defaults, indexes, unique constraints, FKs) is produced once and committed in `fonoteka.go`; a testcontainers test migrates the Go sets up and diffs `information_schema`/`pg_catalog` against it. Intended differences live in a small, commented allow-list in the test. How the snapshot is produced is research; if the PHP migrations do not run cleanly on Postgres, research reports that before planning. **[Answered below — they run cleanly.]**
|
||
- D-03: The Phase 3 minimal tables are widened by one appended ALTER migration per table (`widen_users`, `widen_collections`, `widen_albums`) placed right after the two shipped Phase 3 migrations, followed by the new create-table migrations. Shipped migrations are never edited (P3 D-17). Each widen migration has a real Rollback.
|
||
- D-04: Reference data in the PHP history (the various-artist taxonomy seed, any settings defaults) is ported as idempotent data migrations inside the plugin's set, keyed by natural key, with a Rollback that removes them, same as the genres seed (P3 D-18). No seeder command.
|
||
|
||
**Model DX: fillable, hidden, rules**
|
||
- D-05: Each model declares `Fillable() []string`, a mechanical copy of PHP `$fillable`. `lagoon.Fill` copies onto the model only names that are both requested by the caller and in the model's fillable list. Write services pass their own narrower list exactly as PHP does (`AlbumWriteService::FILL_FIELDS` is a strict subset of `Album::$fillable`: no `collection_id`, no `market_price_source`). The model list is the backstop, the service list is the real boundary. `json.Unmarshal` into a GORM model is never used on a write path (PITFALLS Pitfall 3).
|
||
- D-06: Non-fillable and unknown keys are silently ignored, as Eloquent does; they are never persisted and never produce a 422. In non-production environments the dropped keys are logged once per call site.
|
||
- D-07: Criterion 3 is proven at service level in this phase: the fill boundary of the PHP write services is ported now (at minimum Album, Collection and the four credential models) and fuzzed against real Postgres with random extra and server-owned keys, asserting nothing outside the list is persisted. `lagoon.Fill` has its own framework-level fuzz test on a fixture model. The HTTP-level fuzz over request DTOs is Phase 12's job; criterion 3's "every write endpoint" clause moves there and the planner notes it in the roadmap.
|
||
- D-08: Hidden columns carry `json:"-"` on the GORM model so an accidental marshal cannot leak them, and models also declare `Hidden() []string` (for the Phase 9 admin schema and for a lagoon test that marshals every registered model and asserts hidden names are absent). API payloads are not produced by marshaling models: they come from ported `Serialize*` functions, as PHP's `SerializesFonoteka` does. The "explicit per-call override" is a named, greppable function (`Reveal`-style), not a serializer option.
|
||
- D-09: Models declare `Rules() map[string]string` with the PHP rule strings copied verbatim. `lagoon` translates the Laravel rule grammar Płytarium uses onto go-playground/validator, runs it in a before-save callback, and returns the Laravel-shaped 422 error map with messages translated through `phrasebook`. `unique:table` is a DB check in the same step and must respect soft deletes the way Winter's does. An untranslatable rule fails loudly at boot or test time, never silently passes. **[Full inventory below.]**
|
||
|
||
**Encrypted cast**
|
||
- D-10: The Go cast reads and writes only its own AES-256-GCM format. Laravel CBC payloads are never on the live read path. Phase 5 ships a small, tested helper that decrypts a Laravel `encrypted` payload (base64 JSON with iv/value/mac, AES-256-CBC + HMAC under `APP_KEY`) for the Phase 15 cutover import to re-encrypt; it is not wired into the cast.
|
||
- D-11: The key is `app.key` in config (`SUMMER_APP__KEY`), 32 bytes base64. Missing, short or undecodable key fails boot with no default, same rule as the JWT secret (P3 D-11). The column-encryption key is derived from it with HKDF and a fixed label so `app.key` can serve other purposes later without key reuse. A `summer key:generate` command prints a fresh key.
|
||
- D-12: Ciphertext is versioned: a format/key-id prefix, nonce, ciphertext+tag. Config accepts `app.previous_keys` as decrypt-only keys (Laravel's `APP_PREVIOUS_KEYS` equivalent). No re-encrypt command in this phase.
|
||
- D-13: Encrypted columns are typed `lagoon.Encrypted` on the model: Scanner/Valuer decrypt on read and encrypt on write; `MarshalJSON`, `String()` and `GoString()` always emit a redaction; plaintext is only reachable through an explicit `.Reveal()`. Applies to `api_key` on `UserAiCredential` and `OrgAiCredential` and to the secret columns of `UserDiscogsCredential` and `OrgDiscogsCredential`; research lists the exact columns. **[Confirmed below: `api_key`, `api_key`, `token`, `token` respectively.]**
|
||
|
||
**Attachments (DATA-08)**
|
||
- D-14: Attachments use Winter's `system_files` table shape and name (`disk_name`, `file_name`, `file_size`, `content_type`, `title`, `description`, `field`, `attachment_id`, `attachment_type`, `is_public`, `sort_order`, timestamps). `attachment_type` keeps the PHP class string (`Golem15\Fonoteka\Models\Album`) through a per-model morph name, so cutover copies rows and files verbatim and later Winter ports reuse it. The table and `File` model are framework-owned (`lagoon`) with their own migration set that runs before plugin sets.
|
||
- D-15: Full storage scope lands in this phase (user chose the largest option over the recommended table-only cut): `gocloud.dev/blob` is wired now, files are stored and deleted through it, and thumbnails are generated. Only the HTTP upload endpoint stays in Phase 12. Planner should size the phase accordingly.
|
||
- D-16: v1 bucket is `fileblob` rooted at the same `storage/app/uploads` tree PHP uses, with bucket URL and public path prefix from a `storage.uploads.*` config section mirroring `cms.php`'s `storage.uploads`. Winter's `disk_name` partition path (`public/xxx/yyy/zzz/<disk_name>`) and the relative public URL shape (`relativeMediaUrl`) are reproduced exactly. The framework serves public files with a static handler at the configured path; S3/GCS is a config change later. `memblob` in unit tests.
|
||
- D-17: `Thumb(w, h, mode)` reproduces Winter's thumb filename and location (next to the original, `thumb_<id>_<w>_<h>_0_0_<mode>.<ext>`; research confirms the exact rule). Thumbs copied at cutover are reused, and `thumb_url` stays byte-identical for the parity diff. Pixel equality with PHP GD is not a goal. **[Confirmed below.]**
|
||
- D-18: The resize library is research's pick, recorded here as the phase decision naming the dependency: pure Go, maintained, no cgo/libvips. **[Picked below: `github.com/disintegration/imaging` v1.6.2.]**
|
||
- D-19: Deletion follows Winter: soft-deleting an owner keeps `system_files` rows and blobs; force delete removes rows inside the delete transaction and removes originals and thumbs from blob after commit. This runs through the same lifecycle-callback mechanism as the other cascades (DATA-03), not a special case.
|
||
|
||
### Claude's Discretion
|
||
- Lifecycle hook naming and wiring (Winter names `BeforeValidate`/`BeforeCreate`/`BeforeSave`/`BeforeDelete`/`AfterDelete` as optional interfaces vs. GORM's own hook names), the base model embed, and how soft-delete cascade runs in one transaction.
|
||
- Pivot model shape for `album_artists.sort_order` and `CollectionEditor` (GORM `SetupJoinTable` vs explicit has-many-through), as long as the 3+ artist ordering round-trip and the business columns hold.
|
||
- Money cast type and API: a named string-backed type in `models/` (or `lagoon` if generic) that reproduces `MarketPriceCast::get` (null for null/blank/non-numeric, else 4-decimal string) and leaves normalisation to the `Album` before-save hook, as PHP does. Never `float64` in the JSON path.
|
||
- Jsonable cast shape, including `[]` vs `null` behavior per column (Pitfall 4).
|
||
- Pagination helper API; the envelope is fixed: `{data, meta{current_page,last_page,per_page,total}}`, no `links`.
|
||
- Callback-registry API surface (ARCHITECTURE.md Pattern 3b) and which plugin/model pair demonstrates criterion 5's cross-plugin extension plus companion migration; a fixture plugin in tests is acceptable if no real Płytarium case fits.
|
||
- Soft-delete + unique strategy (partial unique indexes vs. matching PHP's actual constraints); D-02's schema diff decides what "matching" means, the delete-then-recreate test must pass for every soft-deletable uniquely-keyed table. **[Only one real candidate table exists — see below.]**
|
||
- Which `Serialize*` functions are ported now (only those needed by this phase's tests) vs. Phase 12.
|
||
- `Album`'s service-calling hook (`ArtistResolver` in `beforeSave`) becomes a callback registered from `classes/`, per the layout note; Scout/Broadcastable traits are left as seams for Phase 11.
|
||
- CLI-03 details beyond what Phase 3 shipped. **[Almost nothing is left — see below.]**
|
||
- Plan count and split, subject to the plan-count checkpoint and "unit tests are the last plan" rules. Given D-15, expect more than the usual number of plans.
|
||
|
||
### Deferred Ideas (OUT OF SCOPE)
|
||
- HTTP-level fuzz of request DTOs on every write endpoint — Phase 12/13 when the routes exist (second half of criterion 3).
|
||
- Photo upload endpoint, manual cover URL and Discogs cover import — Phase 12 (API-02), HTTP-07, Phase 14.
|
||
- Re-encrypt command for key rotation — after v1; the versioned format (D-12) keeps it possible.
|
||
- Cutover import that decrypts Laravel payloads and re-encrypts, and copies `system_files` rows and the uploads tree — Phase 15, using the helper from D-10.
|
||
- S3/GCS bucket for uploads — config change after cutover.
|
||
- Typesense indexing and Centrifugo broadcasting hooks on `Album` — Phase 11.
|
||
- Correcting the "27 migrations" wording in ROADMAP.md and REQUIREMENTS.md DATA-09, and moving criterion 3's endpoint clause — do at plan time for this phase.
|
||
</user_constraints>
|
||
|
||
<phase_requirements>
|
||
## Phase Requirements
|
||
|
||
| ID | Description | Research Support |
|
||
|----|-------------|------------------|
|
||
| DATA-03 | Timestamps, soft delete, lifecycle hooks, cascading soft delete in a transaction | Per-model inventory below marks every soft-deletable model and every hook (`beforeSave`, `beforeValidate`, `beforeDelete`, `afterDelete`); `Collection.beforeDelete` cascade pattern documented |
|
||
| DATA-04 | belongsTo/hasOne/hasMany/belongsToMany with ordered results and pivot models | Per-model inventory lists every relation; `SetupJoinTable` API verified for `album_artists` (sort_order) and `collection_editors` (role/granted_at/granted_by) |
|
||
| DATA-05 | Rule-string validation via go-playground/validator with Laravel-shaped 422 | Full rule inventory (only 6 models declare `$rules`) with verified tag mapping table |
|
||
| DATA-06 | Fillable allow-list mass assignment; hidden deny-list with explicit override | Per-model fillable/hidden columns extracted verbatim from PHP source |
|
||
| DATA-07 | Jsonable, money (fixed 4-decimal string), encrypted-at-rest casts | `MarketPriceCast` semantics documented verbatim; jsonable column inventory; 4 encrypted columns confirmed |
|
||
| DATA-08 | Polymorphic file attachment via `system_files`, gocloud.dev/blob | `system_files` verified schema; Winter `File` partition/thumb-naming logic read from source; resize library picked (D-18) |
|
||
| DATA-09 | All 25 models + migrations, matching schema | Full 25-model inventory; squashed migration list in FK order; PHP-on-Postgres schema verified live |
|
||
| DATA-10 | Pagination envelope `{data, meta{...}}` no `links` | Confirmed no existing pagination helper in `lagoon`; net-new |
|
||
| DATA-11 | Cross-plugin lifecycle hook + companion migration | GORM callback registry API verified (ARCHITECTURE.md Pattern 3b, confirmed against real GORM docs) |
|
||
| CLI-03 | Migration commands: up, status, rollback last of a named plugin | **Already fully implemented** in `lagoon/commands.go` + `lagoon/migrations.go` since Phase 3/4 — see below |
|
||
</phase_requirements>
|
||
|
||
## Summary
|
||
|
||
Phase 5 ports 25 Płytarium models plus their final-state schema onto GORM/Postgres. The single most important research finding is **D-02 is answered conclusively**: the actual PHP migrations for both `golem15.user` and `golem15.fonoteka` were run against a scratch Postgres 16 container in this session (`php artisan winter:up` against `pgsql`) and completed with zero errors — every one of the 25 Fonoteka tables, `system_files`, `users`, and `golem15_user_organisations` materialized correctly with the exact column types, defaults, and indexes Postgres's own `\d`/`pg_dump` report. This document embeds that verified schema directly, so the planner does not need to re-derive it from reading 38 migration files — the "Full Per-Model Inventory" and "Squashed Migration List" sections below are sourced from live Postgres introspection, not migration-file reading, and are the single source of truth for D-02's snapshot.
|
||
|
||
Second finding: **CLI-03 is already done.** `lagoon.Migrate`, `lagoon.RollbackLast`, `lagoon.Status`, and the `migrate`/`migrate:rollback --plugin`/`migrate:status` bonfire commands were built in Phase 3/4 and already scope correctly to a per-plugin `summer_migrations_<plugin>` history table. This phase's CLI-03 work is verification (a test that `--plugin=fonoteka` rolls back only Fonoteka's last migration, not User's), not new code.
|
||
|
||
Third: only 6 of the 25 models declare Laravel `$rules` (`Album`, `Collection`, `Artist`, `Style`, `Genre`, `Settings`) — DATA-05's real surface area is small. The rule vocabulary in use is `required`, `nullable`, `integer`, `between:X,Y`, `numeric`, `min:X`, `max:X`, `Rule::in([...])`, and `unique:table`; every one maps cleanly onto go-playground/validator (`between` has no direct tag — combine `min`+`max`), verified against the library's own baked-in tag source.
|
||
|
||
Fourth: exactly one table combines soft-delete with a real (non-PK) unique constraint — `golem15_fonoteka_collections.public_token`, and PHP's own migration does **not** use a partial index there. Every other soft-deletable table (`albums`) has no business unique key, and every other uniquely-keyed table (`album_reservations`, `album_ratings`, `wishlist_subscriptions`, `wishlist_digest_queue`, `csv_import_rows`, credential tables) is hard-deleted, not soft-deleted. Pitfall 11's "audit all tables" is a short audit with one real answer.
|
||
|
||
Fifth: the widen migrations named in D-03 are asymmetric in size. `widen_collections` and `widen_albums` need substantial column additions (Albums is currently a 7-column Phase-3 stub; the final table has 28 columns). `widen_users` needs **zero** columns for this phase's own success criteria — no Fonoteka model reads any `users` column beyond `id` — but a **new** table, `golem15_user_organisations`, must be created (structure-only; full Organisation behavior is Phase 7's AUTH-02) because `OrgAiCredential`/`OrgDiscogsCredential` FK into it.
|
||
|
||
**Primary recommendation:** Treat this RESEARCH.md's live-verified schema as authoritative for D-02's snapshot; write the Go migrations directly against it (18 migrations replacing the 38 PHP files, see the squashed list); pick `github.com/disintegration/imaging` v1.6.2 for D-18; model every many-to-many pivot with `SetupJoinTable` and an explicit sync function (never `Association().Append()`) per the verified GORM pivot-column gap.
|
||
|
||
## Architectural Responsibility Map
|
||
|
||
| Capability | Primary Tier | Secondary Tier | Rationale |
|
||
|------------|-------------|----------------|-----------|
|
||
| Model structs, relations, casts, hooks | API/Backend (GORM layer) | — | Pure data-layer concern; no HTTP/SSR involvement in this phase |
|
||
| Migrations (schema DDL) | Database/Storage | API/Backend (gormigrate runner) | Schema lives in Postgres; gormigrate is the authoring/runtime mechanism, owned by the backend process |
|
||
| Mass-assignment fillable/hidden discipline | API/Backend | — | Security boundary enforced in Go model/service code before any HTTP DTO exists (Phase 12 adds the HTTP-facing half) |
|
||
| Rule-string validation | API/Backend | — | Runs in a GORM before-save callback, not client-side |
|
||
| Encrypted-at-rest casts | API/Backend + Database/Storage | — | Encryption/decryption happens in Go (Scanner/Valuer); ciphertext is what's persisted to Postgres |
|
||
| File attachments (`system_files` + blob) | API/Backend | CDN/Static (blob-served public files) | Metadata row owned by backend; actual bytes may later move to S3/GCS (config change, not architecture change) |
|
||
| Pagination envelope | API/Backend | — | Shape emitted by serializers; no client-side pagination logic ported here |
|
||
| Cross-plugin lifecycle hooks (callback registry) | API/Backend | — | GORM's own `db.Callback()` registry, registered at plugin `Boot()` |
|
||
|
||
## Project Constraints (from CLAUDE.md)
|
||
|
||
From the repo-root and `summercms.go` `CLAUDE.md` files (both apply; the `summercms.go`-scoped file is more specific and takes precedence on conflict):
|
||
|
||
- **Never break core plugins** (user, blog, pages, payment) unless directly asked — `widen_users` must be strictly additive; never touch or reinterpret existing `users` columns.
|
||
- **Lean planning**: prefer fewer, larger plans. Given D-15 (full blob storage + thumbnails lands in this phase), expect more plans than a typical phase, but still consolidate where the work is mechanical (e.g., one plan for "the bulk of the 25 models" rather than one plan per model).
|
||
- **Plan-count checkpoint**: present the suggested plan count with a one-line scope each and wait for confirmation before writing PLAN.md files.
|
||
- **Unit tests are always the last plan** of the phase; earlier plans may include smoke tests but are not blocked on full coverage.
|
||
- **Go conventions**: standard library first; GORM, gormigrate, go-playground/validator, gocloud.dev are already-decided per STACK.md; `disintegration/imaging` is this phase's one new named dependency (D-18). `go vet` and `go test ./...` green at every commit. **No AutoMigrate anywhere** (Pitfall 13; this project's own PITFALLS.md is unambiguous, overriding any stray "goose" references elsewhere in older research docs — `gormigrate` is authoritative per STATE.md's Phase 3 decision log).
|
||
- **Compiled plugins only** — no runtime plugin loading; not directly relevant to this phase's model/migration work.
|
||
- **API parity is the acceptance test** — do not "improve" response shapes; this phase's serializers (where ported) must match `SerializesFonoteka` byte-for-byte in the fields it touches.
|
||
- **Commit rules**: no co-author tags; one logical change per commit; planning docs and code in separate commits.
|
||
- **`internal/build`'s models-leaf enforcement** (Phase 4, P4 D-11) applies: `models/` files may only import stdlib + framework + sibling model types, never `classes/`/`controllers/`. The folded todo (`verify-models-leaf-rule.md`) must run before the bulk 25-model port — this is a hard prerequisite task, not optional.
|
||
|
||
## Standard Stack
|
||
|
||
### Core
|
||
| Library | Version | Purpose | Why Standard |
|
||
|---------|---------|---------|--------------|
|
||
| gorm.io/gorm | v1.31.2 | ORM | Already decided (STACK.md), already in `go.mod`/`go.sum` — `[VERIFIED: go.sum]` |
|
||
| github.com/go-gormigrate/gormigrate/v2 | v2.1.7 | Migrations | Already decided and already in use since Phase 3 (`go.mod` line 7) — `[VERIFIED: go.sum]`. **Note:** `slopcheck` flagged this package `[SLOP]` in this session ("created 2 days ago... no source repository linked"); this is a false positive from the sandboxed Go module proxy's "first-seen" timestamp, not the package's actual age — the package has been a real, working, committed dependency in this repo since Phase 3/4 with matching go.sum checksums across both `summercms.go` and `fonoteka.go`. See Package Legitimacy Audit below for the full explanation; it is NOT removed. |
|
||
| github.com/go-playground/validator/v10 | v10.30.4 (released 2026-09-03) | Rule-string validation | Already decided (STACK.md), confirmed current via `proxy.golang.org` — `[VERIFIED: Go module proxy]` |
|
||
| gocloud.dev/blob (+ fileblob, memblob) | v0.46.0 (released 2026-06-02) | Attachment storage abstraction | Already decided (STACK.md, D-15), confirmed current via `proxy.golang.org` — `[VERIFIED: Go module proxy]` |
|
||
| crypto/hkdf (stdlib) | Go 1.27 standard library | HKDF key derivation for `lagoon.Encrypted`'s column key (D-11) | Go 1.24+ ships HKDF in the standard library (`hkdf.Key`/`Extract`/`Expand`) — no `golang.org/x/crypto` dependency needed; this phase's Plan 05-03 uses stdlib directly, correcting an earlier draft that named `golang.org/x/crypto/hkdf` |
|
||
|
||
### Supporting
|
||
| Library | Version | Purpose | When to Use |
|
||
|---------|---------|---------|-------------|
|
||
| github.com/disintegration/imaging | v1.6.2 (tagged 2019-11-16, stable since) | Thumbnail generation (D-18) | `imaging.Fill()` for Winter's `mode=crop`, `imaging.Fit()` for `mode=auto`, `imaging.Resize()` for `mode=exact`. Pure Go, single transitive dependency (`golang.org/x/image`), no cgo. Picked over hand-rolling on raw `golang.org/x/image/draw` (the CONTEXT.md-named baseline) because it already implements exactly the fill/fit/resize semantics Winter's `Resizer` modes need — see Code Examples. |
|
||
| golang.org/x/image | v0.46.0 (latest as of 2026-09-08) | Transitive dependency of `disintegration/imaging` | Pulled in automatically; no direct import needed unless a custom resample filter is required |
|
||
| github.com/testcontainers/testcontainers-go + modules/postgres | v0.44.0 | Real-Postgres tests (D-02 schema-diff test, fuzz tests) | Already in `fonoteka.go/go.mod`; reuse the existing `TestMain` pattern in `lagoon/postgres_test.go` (pl-PL ICU locale) and `fonoteka.go/parity` |
|
||
|
||
### Alternatives Considered
|
||
| Instead of | Could Use | Tradeoff |
|
||
|------------|-----------|----------|
|
||
| `disintegration/imaging` | Hand-rolled crop/fit on `golang.org/x/image/draw` directly | More code to write and test for the exact same result; `imaging` already wraps this correctly and is what CONTEXT.md called "the baseline to beat" — it beats it by being less code, not by being a different approach |
|
||
| `disintegration/imaging` | `github.com/kovidgoyal/imaging` (maintained fork) | Fork adds a `mandykoh/prism` color-management dependency the phase does not need (no ICC profile handling required for simple square-crop thumbnails); upstream `disintegration/imaging` is feature-complete and stable (not archived, 5.7k GitHub stars, zero known CVEs), just not under active new-feature development — acceptable for a "generate a fixed-size crop thumbnail" use case |
|
||
| `SetupJoinTable` for `album_artists` | Plain `many2many` tag (`gorm:"many2many:golem15_fonoteka_album_artists;"`) | Silently drops `sort_order` on every read (Pitfall 12, confirmed against GORM's own docs: the default many2many API has no path to extra pivot columns without `SetupJoinTable`) |
|
||
| Custom `MarketPriceCast`-equivalent | GORM's `serializer:json` or a raw `decimal.Decimal` (shopspring) type | PHP's own docblock on `MarketPriceCast` explains why a generic decimal library is wrong here: it must accept a blank string as "clear the price" without throwing, and always read back as a stable 4-decimal string, never a float — the same reasoning applies to any Go decimal library with the same blank-string/`sql.Scanner` trap |
|
||
|
||
**Installation:**
|
||
```bash
|
||
go get github.com/disintegration/imaging@v1.6.2
|
||
```
|
||
(gorm, gormigrate, validator/v10, gocloud.dev are already present in `go.mod`.)
|
||
|
||
## Package Legitimacy Audit
|
||
|
||
| Package | Registry | Age (real, via proxy.golang.org) | Downloads/Stars | Source Repo | slopcheck | Disposition |
|
||
|---------|----------|-----------------------------------|------------------|-------------|-----------|-------------|
|
||
| github.com/disintegration/imaging | Go module proxy | Tagged v1.6.2 in 2019 (6+ yrs), repo pushed as recently as 2023, not archived | 5,756 GitHub stars | github.com/disintegration/imaging | [OK] | Approved |
|
||
| github.com/go-gormigrate/gormigrate/v2 | Go module proxy | Real age: multi-year, already shipped in this repo's `go.sum` since Phase 3 | 1.2k GitHub stars (per STACK.md) | github.com/go-gormigrate/gormigrate | **[SLOP]** (false positive — see note) | **Approved (override)** — already a locked, working, in-use dependency; the SLOP verdict is an artifact of `slopcheck`'s Go-ecosystem check using the sandboxed module proxy's "first indexed" timestamp rather than the package's real registration date. Do not act on this flag; do not remove. |
|
||
| github.com/go-playground/validator/v10 | Go module proxy | Real release date 2026-09-03 (confirmed), library itself is 10+ years old (established since ~2015) | Long-established, already decided in STACK.md | github.com/go-playground/validator | [SUS] (false positive, same proxy-timestamp cause) | Approved — already-decided, HIGH confidence per STACK.md's own `pkg.go.dev`-sourced verification |
|
||
| golang.org/x/image | Go module proxy | Official Go team subrepo | Official golang.org/x org | golang.org/x/image | [SUS] (false positive, same cause) | Approved — official Go team package, transitive via `imaging` |
|
||
| gocloud.dev | Go module proxy | Real release 2026-06-02 | Google-maintained | github.com/google/go-cloud | [OK] | Approved (already decided) |
|
||
|
||
**Packages removed due to slopcheck [SLOP] verdict:** none — see override note above.
|
||
**Packages flagged as suspicious [SUS]:** none requiring a `checkpoint:human-verify` — all three SUS flags in this run are the same Go-ecosystem proxy-timestamp false positive on well-established, already-decided-elsewhere packages (STACK.md independently verified these via `pkg.go.dev` with HIGH confidence). `disintegration/imaging` — the one genuinely new package this research introduces — came back clean `[OK]`.
|
||
|
||
**Caveat for future phases:** `slopcheck`'s `--ecosystem go` check appears to key package age off the sandboxed environment's Go module proxy cache "first seen" timestamp, not the upstream repository's actual creation/release date. This is a poor signal for Go modules specifically (unlike npm/PyPI, where registry publish date is reliable) and produced 3 false positives out of 5 packages checked in this session, including one already-shipped dependency. Future phase research should cross-check any Go `[SLOP]`/`[SUS]` verdict against `proxy.golang.org/<module>/@v/<version>.info` (real `Time` field) and the module's actual GitHub star count/repo age before acting on it.
|
||
|
||
**Side-effect warning:** `slopcheck install ... --force` in this environment actually ran `go get` and modified `go.mod`/`go.sum` as a side effect of the "check-then-install" command shape. This was caught and reverted (`git checkout -- go.mod go.sum`) before any other work in this session. Future research sessions should use a scratch/throwaway module (or a `go.mod` copy) when running `slopcheck install --force` against a real project's dependency file, or prefer `slopcheck scan` if it exists as a non-installing alternative.
|
||
|
||
## Architecture Patterns
|
||
|
||
### System Architecture Diagram
|
||
|
||
```
|
||
Plugin Boot (fonoteka.Boot / user.Boot)
|
||
│
|
||
├─→ pact.HasMigrations.Migrations() ──→ lagoon.Migrate ──→ gormigrate per-plugin history table ──→ Postgres DDL
|
||
│ (this phase: ~18 Go migrations, FK-ordered, see Squashed Migration List)
|
||
│
|
||
├─→ pact.HasModels.Models() ──→ registered with GORM for schema reflection / callback targeting
|
||
│ (this phase: 25 Fonoteka models + companion structs for user.User / user.Organisation)
|
||
│
|
||
└─→ db.Callback().Create()/.Update()/.Delete().After(...).Register(...) ──→ cross-plugin lifecycle hooks (DATA-11)
|
||
|
||
Write path (service layer, this phase's tests only — no HTTP yet):
|
||
caller (test / fuzz harness)
|
||
→ lagoon.Fill(model, fillableAllowList, requestMap) [D-05/D-06: silently drops non-fillable/unknown keys]
|
||
→ model.BeforeValidate() / BeforeSave() (Winter-equivalent hooks, DATA-03)
|
||
→ lifeguard.Validate(model.Rules(), values) [D-09: validator.Var() per field + unique:table DB check]
|
||
→ tx.Save(model) [GORM callbacks fire — DATA-11 cross-plugin hooks]
|
||
→ tx.Commit()
|
||
|
||
Read/serialize path (this phase's tests only):
|
||
model loaded via GORM (relations preloaded with ORDER BY for belongsToMany/hasMany — DATA-04)
|
||
→ Serialize*(model) [ported subset of SerializesFonoteka — hidden columns never touched, json:"-" backstop]
|
||
→ paginate(query) → {data, meta{current_page,last_page,per_page,total}} [DATA-10, no `links`]
|
||
|
||
Attachment path (DATA-08, D-15):
|
||
File{} (framework-owned, system_files table)
|
||
→ blob.Bucket (gocloud.dev/blob, fileblob rooted at storage/app/uploads in dev/prod, memblob in tests)
|
||
→ disk_name partition path: first 9 chars of disk_name split into 3×3 groups ("abc/123/xyz/")
|
||
→ Thumb(w,h,mode) → thumb_<id>_<w>_<h>_0_0_<mode>.<ext>, generated lazily via disintegration/imaging on first call
|
||
→ deletion: soft-delete owner keeps file row+blob; force delete removes row in-tx, blob+thumbs after commit (D-19)
|
||
```
|
||
|
||
### Recommended Project Structure (fonoteka.go, per the Winter-directory layout note)
|
||
```
|
||
plugins/golem15/fonoteka/
|
||
├── models/ # 25 model structs + Fillable()/Hidden()/Rules() — leaf package (internal/build enforced)
|
||
├── classes/ # ArtistResolver, AlbumWriteService port, service-calling hooks (Album.beforeSave's ArtistResolver call)
|
||
├── classes/casts/ # MarketPriceCast-equivalent (if not promoted to lagoon)
|
||
├── updates/ # ~18 squashed gormigrate migrations, FK-ordered
|
||
├── plugin.go # Models()/Migrations() registration
|
||
plugins/golem15/user/
|
||
├── models/ # User (existing stub, widened per this phase's findings — likely no column changes needed)
|
||
│ # Organisation (new companion struct, structure-only)
|
||
├── updates/ # widen_users (likely no-op/minimal) + create_organisations (new)
|
||
lagoon/
|
||
├── attach/ # NEW: File model (system_files), blob wiring, Thumb() — framework-owned per D-14
|
||
├── fill.go # NEW: lagoon.Fill (D-05/D-06)
|
||
├── encrypted.go # NEW: lagoon.Encrypted cast type (D-10..D-13)
|
||
├── validate.go # NEW: rule-string → validator.Var() translation + 422 map (D-09) — or a dedicated `lifeguard` package if that's already scaffolded
|
||
├── paginate.go # NEW: {data, meta{...}} envelope helper (DATA-10)
|
||
```
|
||
|
||
### Pattern: GORM many-to-many with pivot business columns (DATA-04, Pitfall 12)
|
||
**What:** `Album.artists` (via `golem15_fonoteka_album_artists`, columns `album_id, artist_id, sort_order`) and `Collection.editors` (via `golem15_fonoteka_collection_editors`, columns `id, collection_id, user_id, role, granted_at, granted_by, created_at, updated_at`) both carry business data on the pivot that GORM's default `many2many` tag silently drops.
|
||
**When to use:** Any `belongsToMany` in the PHP source with a `'pivot' => [...]` key (both cases in this phase) or an `'order' => ...` key.
|
||
**Example (verified against GORM's own docs):**
|
||
```go
|
||
// Source: https://gorm.io/docs/many_to_many.html (SetupJoinTable API)
|
||
type AlbumArtist struct {
|
||
AlbumID uint `gorm:"column:album_id;primaryKey"`
|
||
ArtistID uint `gorm:"column:artist_id;primaryKey"`
|
||
SortOrder int `gorm:"column:sort_order;default:0"`
|
||
}
|
||
func (AlbumArtist) TableName() string { return "golem15_fonoteka_album_artists" }
|
||
|
||
// At boot/migration-registration time:
|
||
err := db.SetupJoinTable(&Album{}, "Artists", &AlbumArtist{})
|
||
```
|
||
**Critical gap (undocumented by GORM, confirmed live in this session's research):** GORM's `Association("Artists").Append()`/`Replace()` do not provide a documented path to set `sort_order` during the association write. **Do not rely on Association Mode for pivot writes.** Instead, port `AlbumWriteService::syncArtists()`'s behavior as an explicit function: delete-then-bulk-insert (or `ON CONFLICT DO UPDATE`) directly against the join table within the same transaction as the parent save, exactly mirroring PHP's `$album->artists()->sync($idsWithSortOrder)`. Query ordering for reads uses `ORDER BY sort_order` (Artists) / the pivot's own columns (CollectionEditor), not relation-preload magic.
|
||
|
||
### Pattern: Cross-plugin lifecycle extension without editing the owning model (DATA-11)
|
||
**What:** GORM's own callback registry lets `fonoteka` (or a later demo plugin) hook another plugin's model lifecycle without importing/editing its file.
|
||
**Example (from ARCHITECTURE.md, cross-checked against GORM's `Callback()` API shape which is a real, documented GORM feature):**
|
||
```go
|
||
db.Callback().Create().After("gorm:create").Register("fonoteka:notify_album_created", func(tx *gorm.DB) {
|
||
if tx.Statement.Schema == nil || tx.Statement.Schema.ModelType != reflect.TypeOf(Album{}) {
|
||
return
|
||
}
|
||
// ... side effect, registered once at boot
|
||
})
|
||
```
|
||
Pair this with ARCHITECTURE.md Pattern 3c (companion struct + own migration) for the "extend a model's **schema**" half of criterion 5 — e.g. a fixture/demo plugin adds a column to `golem15_fonoteka_albums` via its own migration and queries it through its own struct embedding `fonoteka.Album`, never touching `fonoteka`'s file.
|
||
|
||
### Pattern: Encrypted-at-rest cast (D-10..D-13)
|
||
The PHP `casts = ['api_key' => 'encrypted']` / `['token' => 'encrypted']` idiom becomes a `lagoon.Encrypted` `sql.Scanner`/`driver.Valuer` type:
|
||
```go
|
||
type UserAiCredential struct {
|
||
ID uint `gorm:"primaryKey"`
|
||
UserID uint `gorm:"column:user_id"`
|
||
Provider string `gorm:"column:provider"`
|
||
APIKey lagoon.Encrypted `gorm:"column:api_key"` // AES-256-GCM, redacts on String()/MarshalJSON()
|
||
Model *string `gorm:"column:model"`
|
||
BaseURL *string `gorm:"column:base_url"`
|
||
}
|
||
func (UserAiCredential) Hidden() []string { return []string{"api_key"} } // D-08 backstop even though json:"-" already prevents marshal
|
||
```
|
||
Column is `text` in Postgres (never `varchar`) — confirmed live: both `api_key` and `token` columns are `text`, matching PHP's own docblock reasoning ("ciphertext routinely exceeds 200 chars").
|
||
|
||
### Pattern: Money cast (never float64)
|
||
```go
|
||
// Source: PHP classes/casts/MarketPriceCast.php, read directly, and Pitfall 5
|
||
type MoneyString string // named type, GORM/JSON round-trips as a plain string
|
||
|
||
// Scanner: reads numeric(10,4) from Postgres, formats as 4-decimal string, "" on NULL.
|
||
// Valuer: passes the raw string through — Album.BeforeSave() owns ALL normalization
|
||
// (blank→NULL, currency default, checked_at stamping), matching PHP's division of labor
|
||
// where MarketPriceCast::set() is a pure passthrough and Album::beforeSave() does the work.
|
||
```
|
||
Column: `numeric(10,4)` nullable (confirmed live schema). Validation rule: `nullable|numeric|min:0|max:999999.9999` — the `max` bound must be checked against the **parsed numeric value**, not the formatted string, since go-playground/validator's `max` tag does not parse arbitrary decimal-string types automatically (register a custom validation func, or validate against a `float64`/`decimal` intermediate before the cast normalizes it to the fixed string).
|
||
|
||
### Anti-Patterns to Avoid
|
||
- **Reusing `Association().Append()` for pivot writes with extra columns:** silently loses `sort_order`/`role`/`granted_at`/`granted_by` — write the join table directly (see pattern above).
|
||
- **`AutoMigrate` for anything, including test-only schemas:** this project's own PITFALLS.md (Pitfall 13) and STATE.md Phase 3 decision log are explicit and override any older "goose" references in ARCHITECTURE.md/PITFALLS.md prose — `gormigrate` with real up/down `.Migrate()`/`.RollbackLast()` files is the only mechanism, everywhere, including tests.
|
||
- **Validating the Money field's `max` bound against the formatted string:** `"999999.9999"` compared lexicographically against `"1000000.0000"` gives the wrong answer — validate the numeric value, format afterward.
|
||
- **Treating `golem15_user_organisations` as a Fonoteka-owned table:** it is logically owned by `golem15.user` (mirrors the real PHP plugin that created it) even though only `fonoteka`'s credential tables use it in this phase; its create migration belongs in the `user` plugin's migration set, not `fonoteka`'s, keeping D-03's per-plugin ownership rule intact.
|
||
|
||
## Full Per-Model Inventory
|
||
|
||
**Source of truth:** live Postgres 16 schema, produced by running the actual PHP `winter:up` migration command against a scratch `postgres:16-alpine` container in this research session (see "D-02 Verified" below for the exact method). Cross-checked against direct reads of all 25 PHP model files and all 38 migration files. All facts in this table are `[VERIFIED: live Postgres introspection]` unless noted.
|
||
|
||
Legend: **F**=Fillable, **H**=Hidden, **J**=Jsonable, **C**=Casts, **SD**=Soft Delete, **U**=Unique (besides PK)
|
||
|
||
| # | Model | Table | F | H | J / C | Relations | Hooks | SD | U |
|
||
|---|-------|-------|---|---|-------|-----------|-------|----|----|
|
||
| 1 | Album | `golem15_fonoteka_albums` (28 cols) | name, collection_id, shelf, genre_id, quantity, notes, artist_display, year, format, condition, barcode, discogs_id, edition, tracklist, cover_import_failures, label, catalog_number, country, market_price_stored, market_price_currency, market_price_source | — | J: tracklist, cover_import_failures. C: market_price_stored→MarketPriceCast | belongsTo collection, genre; hasMany ratings; hasOne reservation (unique album_id); belongsToMany styles (order:name), artists (pivot:sort_order, order:sort_order); attachMany photos | beforeSave: refreshTrackTitles + stampMarketPrice | Yes | none (besides PK) |
|
||
| 2 | AlbumRating | `golem15_fonoteka_album_ratings` | album_id, user_id, rating | — | — | belongsTo album, user | — | No | (album_id, user_id) |
|
||
| 3 | AlbumReservation | `golem15_fonoteka_album_reservations` | album_id, user_id, reserved_at, revealed_at | — | — | belongsTo user, album | — | No | album_id |
|
||
| 4 | ApiToken | `golem15_fonoteka_api_tokens` | user_id, name, token_hash, scopes, collection_ids, expires_at, revoked_at, last_used_at, last_used_ip, oauth_client_id | token_hash | J: scopes, collection_ids | belongsTo user | — | No | token_hash |
|
||
| 5 | Artist | `golem15_fonoteka_artists` | name, name_key, slug, discogs_artist_id, is_various | — | — | belongsToMany albums (pivot:sort_order, order:name) | beforeValidate: slug+name_key defaulting | No | name_key |
|
||
| 6 | CollectionEditor (Pivot) | `golem15_fonoteka_collection_editors` | (pivot, not user-facing) | — | dates:granted_at | belongsTo via Collection.editors pivotModel | — | No | (collection_id, user_id) |
|
||
| 7 | CollectionInvitation | `golem15_fonoteka_collection_invitations` | collection_id, email, token_hash, invited_by, expires_at, accepted_at, accepted_by, revoked_at | — | — | belongsTo collection, inviter(User), acceptedUser(User) | computed `status` accessor (accepted/revoked/expired/pending) | No | token_hash |
|
||
| 8 | Collection | `golem15_fonoteka_collections` (12 cols) | name, description, owner_id | public_token | — | belongsTo owner(User); hasMany albums; attachMany photos; attachOne image; belongsToMany editors (pivot:role,granted_at,granted_by; pivotModel:CollectionEditor) | beforeDelete: cascade soft-delete every album in a DB transaction | Yes | public_token (nullable) |
|
||
| 9 | CsvImport | `golem15_fonoteka_csv_imports` | user_id, collection_id, match_job_id, import_job_id, status, import_mode, column_map, original_filename, storage_path, row_count, error_message | — | J: column_map | belongsTo user, collection; hasMany rows | — | No | none |
|
||
| 10 | CsvImportRow | `golem15_fonoteka_csv_import_rows` | csv_import_id, row_index, status, raw_json, artist, title, candidates_json, selected_discogs_id, matched_album_id, draft_json, error_code, error_message | — | J: raw_json, candidates_json, draft_json | belongsTo import(CsvImport), matchedAlbum(Album) | — | No | (csv_import_id, row_index) |
|
||
| 11 | Genre | `golem15_fonoteka_genres` | name, slug, description | — | — | hasMany albums | beforeValidate: slug default; afterDelete: unassign (not cascade-delete) albums | No | none |
|
||
| 12 | Notification | `golem15_fonoteka_notifications` | user_id, type, payload, read_at | — | J: payload | (none declared; no FK either) | — | No | none |
|
||
| 13 | OAuthAuthCode | `golem15_fonoteka_oauth_auth_codes` | request_id, code_hash, client_id, user_id, redirect_uri, scopes, collection_ids, code_challenge, code_challenge_method, resource, state, expires_at, used_at, offline_access | code_hash | J: scopes, collection_ids (stored as `text`, not `json` column type — PHP jsonable on a text column) | belongsTo user | — | No | request_id, code_hash |
|
||
| 14 | OAuthClient | `golem15_fonoteka_oauth_clients` | client_id, client_secret_hash, client_name, redirect_uris, grant_types, token_endpoint_auth_method, registration_ip, consented_at, revoked_at, scope_ceiling | client_secret_hash | J: redirect_uris, grant_types, scope_ceiling (all `text` columns) | (none) | — | No | client_id |
|
||
| 15 | OAuthRefreshToken | `golem15_fonoteka_oauth_refresh_tokens` | token_hash, api_token_id, client_id, user_id, scopes, collection_ids, expires_at, revoked_at, rotated_to_id, offline_access | token_hash | J: scopes, collection_ids (`text`) | belongsTo user, apiToken(ApiToken) | — | No | token_hash |
|
||
| 16 | OrgAiCredential | `golem15_fonoteka_org_ai_credentials` | organisation_id, provider, api_key, model, base_url | api_key | C: api_key→encrypted | belongsTo organisation(Organisation, in `golem15.user`) | — | No | organisation_id |
|
||
| 17 | OrgDiscogsCredential | `golem15_fonoteka_org_discogs_credentials` | organisation_id, token | token | C: token→encrypted | belongsTo organisation | — | No | organisation_id |
|
||
| 18 | PendingInvitationRegistration | `golem15_fonoteka_pending_invitation_registrations` | user_id, invitation_id | — | — | belongsTo user | — | No | user_id |
|
||
| 19 | Settings | *(no dedicated table — Winter `System.Behaviors.SettingsModel` over `system_settings`, keyed by `settingsCode='golem15_fonoteka_settings'`)* | search_use_typesense | — | — | — | — | No | n/a |
|
||
| 20 | Style | `golem15_fonoteka_styles` | name, slug, description | — | — | belongsToMany albums (order:name) | beforeValidate: slug (with min-length-3 collision-safe fallback); afterDelete: detach pivot | No | none |
|
||
| 21 | UserAiCredential | `golem15_fonoteka_user_ai_credentials` | user_id, provider, api_key, model, base_url | api_key | C: api_key→encrypted | belongsTo user | — | No | user_id |
|
||
| 22 | UserCollectionContext | `golem15_fonoteka_user_collection_contexts` | user_id, collection_id | — | — | belongsTo user, collection | — | No | user_id |
|
||
| 23 | UserDiscogsCredential | `golem15_fonoteka_user_discogs_credentials` | user_id, token | token | C: token→encrypted | belongsTo user | — | No | user_id |
|
||
| 24 | WishlistDigestQueue | `golem15_fonoteka_wishlist_digest_queue` | user_id, collection_id, item_count | — | — | (none declared) | static `enqueue()` upsert helper (application-level, not a model hook) | No | (user_id, collection_id) |
|
||
| 25 | WishlistSubscription | `golem15_fonoteka_wishlist_subscriptions` | user_id, collection_id, ws_enabled, email_enabled, subscribed_at | — | — | belongsTo user, collection | — | No | (user_id, collection_id) |
|
||
|
||
**Rules() inventory (DATA-05) — only these 6 models declare `$rules`:**
|
||
| Model | Field | PHP rule string | go-playground/validator mapping |
|
||
|-------|-------|------------------|----------------------------------|
|
||
| Album | name | `required` | `required` |
|
||
| Album | year | `nullable\|integer\|between:1889,2100` | `omitempty,min=1889,max=2100` on `*int` (nil pointer = PHP null, skipped; non-nil zero still validated) |
|
||
| Album | market_price_stored | `nullable\|numeric\|min:0\|max:999999.9999` | Custom func (validator's built-in `min`/`max` don't parse a string-backed Money type) validating the parsed `float64` before cast normalization; `numeric` tag on the raw string input works if validating pre-cast |
|
||
| Album | format | `nullable` + `Rule::in(FORMATS)` (incl. `EP 7"` — contains a space and a `"`) | `omitempty,oneof='EP 7"' LP 2LP CD 2CD MC Box` — verified: `oneof`'s parser strips single-quote delimiters, so a value containing a literal `"` inside single quotes is safe |
|
||
| Album | condition | `nullable` + `Rule::in(CONDITIONS)` | `omitempty,oneof=M NM VG+ VG G` — **`VG+` contains no space, no quoting needed** |
|
||
| Album | market_price_currency | `nullable` + `Rule::in(MARKET_CURRENCIES)` | `omitempty,oneof=USD GBP EUR CAD AUD JPY CHF MXN BRL NZD SEK ZAR` |
|
||
| Album | market_price_source | `nullable` + `Rule::in(MARKET_PRICE_SOURCES)` | `omitempty,oneof=suggestion lowest` |
|
||
| Collection | name | `required` | `required` |
|
||
| Artist | name | `required` | `required` |
|
||
| Style | name | `required` | `required` |
|
||
| Style | slug | `required\|between:3,64\|unique:golem15_fonoteka_styles` | `required,min=3,max=64` + separate DB `unique:table` check step |
|
||
| Genre | name | `required` | `required` |
|
||
| Settings | search_use_typesense | `boolean` | No native `boolean` validator tag; redundant with Go's `bool` type if the field is typed — only relevant if validating an untyped `any` from a YAML/JSON payload before assignment, in which case a small custom check (`_, ok := v.(bool)`) suffices |
|
||
|
||
**`between` has no built-in go-playground/validator tag** — verified directly against the library's `doc.go`. Always translate `between:X,Y` to `min=X,max=Y` (works identically for numeric ranges and, separately, for string/slice length — `go-playground/validator` overloads `min`/`max` by field kind).
|
||
|
||
**`unique:table` is never a struct-tag validator concern** — it is a DB existence query, run in the same before-save step, scoped to exclude soft-deleted rows on tables that have `deleted_at` (none of the 6 rule-bearing models are soft-deletable except `Collection`, which has no `unique:` rule, so this scoping concern is currently moot for the 6-model inventory but must still be built generically per D-09's wording, since a future plugin's rules could combine both).
|
||
|
||
## D-02 Verified: PHP Migrations Run Cleanly on Postgres
|
||
|
||
**Method (reproducible):**
|
||
```bash
|
||
docker run -d --name fonoteka-parity-pg -e POSTGRES_PASSWORD=parity -e POSTGRES_USER=parity \
|
||
-e POSTGRES_DB=fonoteka_parity -p 15432:5432 postgres:16-alpine
|
||
cd /path/to/fonoteka # the PHP repo
|
||
DB_CONNECTION=pgsql DB_HOST=127.0.0.1 DB_PORT=15432 DB_DATABASE=fonoteka_parity \
|
||
DB_USERNAME=parity DB_PASSWORD=parity php artisan winter:up
|
||
```
|
||
**Result:** every migration for `Golem15.User` (1.0.1 → 3.4.1) and `Golem15.Fonoteka` (1.0.1 → 1.2.6) completed with `DONE`, including the destructive `MigrateItemsToAlbums` cutover migration (which has real assertion/verification logic, not just DDL) and every idempotent-guard (`Schema::hasTable`/`hasColumn`) migration. `pdo_pgsql` was already installed; no code changes were needed. `config/database.php`'s `pgsql` connection config works out of the box (`sslmode=prefer`, `search_path=public`).
|
||
|
||
**The only failure** was in an unrelated plugin (`Golem15.Journal`, not in this phase's scope) trying to reach a Typesense server at `127.0.0.1:8181` — a network dependency of that plugin's demo-data seeder, not a schema issue. Fonoteka and User's schemas fully materialized before that failure.
|
||
|
||
**Recommendation for the actual D-02 schema-diff test infrastructure:** reproduce this exact method (`docker run postgres:16-alpine` + `php artisan winter:up` against it) as a **one-time, offline snapshot generation step**, not a live CI dependency on the PHP codebase (the PHP repo is a sibling directory outside `fonoteka.go`'s module, and CI should not need PHP/Composer installed). Capture the output of:
|
||
```bash
|
||
pg_dump -U parity -d fonoteka_parity --schema-only \
|
||
-t 'golem15_fonoteka_*' -t 'system_files' -t 'users' -t 'golem15_user_organisations' \
|
||
--no-owner --no-privileges > fonoteka.go/parity/testdata/php_schema_snapshot.sql
|
||
```
|
||
and commit the resulting SQL (or a normalized JSON extraction from `information_schema`) as the D-02 golden snapshot. This exact schema (all 25 tables + `system_files` + `users` + `golem15_user_organisations`) is embedded in the Full Per-Model Inventory above and is `[VERIFIED: live Postgres introspection, this session]`.
|
||
|
||
## Squashed Migration List (FK order)
|
||
|
||
Replacing 38 PHP migration files with ~18 Go migrations, per D-01. Comment each with the PHP update files it folds.
|
||
|
||
| # | Migration | Table(s) | Folds PHP files | Depends on |
|
||
|---|-----------|----------|------------------|------------|
|
||
| 0 | `golem15.summercms` framework: `create_system_files` | `system_files` | `modules/system/database/migrations/2013_10_01_000002_Db_System_Files.php` + `2025_04_10_000031...metadata.php` | none — runs before every plugin set (D-14) |
|
||
| 1 | `user`: `widen_users` | `users` | *(no PHP files fold in — see finding below; likely empty/no-op for this phase)* | Phase 3's `create_users` |
|
||
| 2 | `user`: `create_organisations` | `golem15_user_organisations` | `v3.2.0/create_organisations_table.php` (structure only; full behavior is Phase 7 AUTH-02) | `widen_users` (ordering only, no real dependency) |
|
||
| 3 | `fonoteka`: `widen_collections` | `golem15_fonoteka_collections` | `v1.1.5/add_public_share_to_collections.php`, `v1.2.4/add_reservations_allowed_to_collections.php`, plus `description` (was always present, verify it's in Phase 3's create) | Phase 3's `create_schema` |
|
||
| 4 | `fonoteka`: `widen_albums` | `golem15_fonoteka_albums` | `v1.1.0/add_music_fields_to_albums.php`, `v1.1.1/add_cover_import_failures...`, `v1.1.4/add_album_catalog_completeness...`, `v1.1.8/add_album_sync_indexes...`, `v1.2.5/add_market_price_to_albums...`, `v1.2.6/add_market_price_source...` | Phase 3's `create_schema` |
|
||
| 5 | `fonoteka`: `create_artists` | `golem15_fonoteka_artists`, `golem15_fonoteka_album_artists` (+ seed Various Artists) | `v1.1.0/create_artists_tables.php`, `v1.1.0/seed_genre_and_various_artist_taxonomy.php` (artist half) | `widen_albums` |
|
||
| 6 | `fonoteka`: `create_styles` | `golem15_fonoteka_styles`, `golem15_fonoteka_album_styles` | (styles table is the renamed-and-final `tags`/`album_styles`; author fresh, matching final shape) | `widen_albums` |
|
||
| 7 | `fonoteka`: `create_album_ratings` | `golem15_fonoteka_album_ratings` | `v1.1.0/create_album_ratings_table.php` | `widen_albums` |
|
||
| 8 | `fonoteka`: `create_album_reservations` | `golem15_fonoteka_album_reservations` | `v1.2.4/create_album_reservations_table.php` | `widen_albums` |
|
||
| 9 | `fonoteka`: `create_collection_invitations` | `golem15_fonoteka_collection_invitations` | `v1.0.8/create_collection_invitations_table.php` | `widen_collections` |
|
||
| 10 | `fonoteka`: `create_pending_invitation_registrations` | `golem15_fonoteka_pending_invitation_registrations` | `v1.0.8/create_pending_invitation_registrations_table.php` | `create_collection_invitations` |
|
||
| 11 | `fonoteka`: `create_api_tokens` | `golem15_fonoteka_api_tokens` | `v1.0.3/create_fonoteka_api_tokens_table.php`, `v1.0.6/add_area_ids...` (renamed to collection_ids), `v1.1.7/add_oauth_client_id...` | none new (users only) |
|
||
| 12 | `fonoteka`: `create_oauth_tables` | `golem15_fonoteka_oauth_clients`, `_oauth_auth_codes`, `_oauth_refresh_tokens` | `v1.1.7/create_oauth_tables.php`, `v1.1.9/add_scope_ceiling...` | `create_api_tokens` (refresh_tokens FK) |
|
||
| 13 | `fonoteka`: `create_csv_import_tables` | `golem15_fonoteka_csv_imports`, `_csv_import_rows` | `v1.1.2/create_csv_import_tables.php`, `v1.1.6/add_csv_import_mode.php` | `widen_collections`, `widen_albums` |
|
||
| 14 | `fonoteka`: `create_credentials_tables` | `_user_ai_credentials`, `_org_ai_credentials`, `_user_discogs_credentials`, `_org_discogs_credentials` | `v1.0.4`, `v1.0.5`, `v1.1.3` (both files) | `create_organisations` (org FK) |
|
||
| 15 | `fonoteka`: `create_notifications` | `golem15_fonoteka_notifications` | `v1.2.1/create_notifications_table.php` | none new |
|
||
| 16 | `fonoteka`: `create_wishlist_subscriptions` | `golem15_fonoteka_wishlist_subscriptions` | `v1.2.2/create_wishlist_subscriptions_table.php` | `widen_collections` |
|
||
| 17 | `fonoteka`: `create_wishlist_digest_queue` | `golem15_fonoteka_wishlist_digest_queue` | `v1.2.3/create_wishlist_digest_queue_table.php` | `widen_collections` |
|
||
|
||
**Genres table needs no migration in this phase** — Phase 3's `create_schema` + `seed_genres` already produced the final shape (verified: live schema for `golem15_fonoteka_genres` is `id, name, slug, description, created_at, updated_at` with a `slug` index — byte-identical to Phase 3's migration).
|
||
|
||
**`collection_editors` and `user_collection_contexts` tables need no widening** — Phase 3's `create_schema` already created them in their exact final shape (verified against live schema: `role`/`granted_at`/`granted_by`/unique(collection_id,user_id) and unique(user_id) respectively already match).
|
||
|
||
### `widen_users` finding
|
||
No Fonoteka model's fillable/hidden/casts/relations reads any `users` column beyond `id` (FK target only). The real, final PHP `users` table has 52 columns (2FA, GDPR consent, OAuth social login, device auth, etc.) — none of which any of the 25 Fonoteka models touch. **This phase's `widen_users` migration can legitimately be empty or omitted entirely** unless the planner chooses to pre-add `organisation_id`/`organisation_role` now (cheap, additive, unblocks Phase 7 from a second users-widening pass) — flagged as an **open question** below rather than decided here, since CONTEXT.md named `widen_users` as one of three concrete widen targets without specifying its content, and the phase boundary text ("plus the golem15.user tables they depend on") suggests only FK-target existence is required, not column parity with the eventual full User model.
|
||
|
||
### Soft-delete + unique audit (Pitfall 11 — the whole audit, not a sample)
|
||
Checked all 25+ tables (`deleted_at` presence × non-PK unique constraint presence), verified against live schema:
|
||
|
||
| Table | Has `deleted_at`? | Has non-PK unique constraint? | Conflict? |
|
||
|-------|---|---|---|
|
||
| `golem15_fonoteka_albums` | Yes | No | No — nothing to protect |
|
||
| `golem15_fonoteka_collections` | Yes | Yes: `public_token` (nullable) | **Yes — the one real case.** PHP's own migration uses a **plain** `UNIQUE` constraint, no partial index (`WHERE deleted_at IS NULL`). Match PHP exactly (plain unique) for parity; the practical collision risk is negligible (32-char random token), but the delete-then-recreate test (success criterion 5) should target this column specifically: soft-delete a Collection with a set `public_token`, then attempt to set the same literal token value on a new/different Collection, and assert PHP's actual behavior (constraint violation) is reproduced — this is "matching PHP's actual constraints" per the Claude's-Discretion note, not "fixing" it with a partial index PHP doesn't have. |
|
||
| All other 23 tables | No `deleted_at` (hard-deleted or never deleted) | various | No — soft-delete + unique intersection doesn't apply |
|
||
|
||
**Practical implication:** the success-criterion-5 "delete-then-recreate test... for every soft-deletable uniquely-keyed table" has exactly **one** target table (`golem15_fonoteka_collections.public_token`) in the real schema, not an open-ended set. Size the plan accordingly.
|
||
|
||
## CLI-03: Already Implemented
|
||
|
||
`lagoon.Migrate(gdb, plugins)`, `lagoon.RollbackLast(gdb, plugins, pluginID)`, `lagoon.Status(gdb, plugins)`, and the `migrate`/`migrate:rollback --plugin`/`migrate:status` bonfire commands (`lagoon/commands.go`) were built in Phase 3/4 and are functionally complete: `RollbackLast` resolves an empty `--plugin` flag to the last-migrated plugin, validates the plugin is activated and has migrations, and calls gormigrate's own `RollbackLast()` against that plugin's isolated `summer_migrations_<plugin>` history table (via `HistoryTableName`). This **already satisfies** REQUIREMENTS.md's CLI-03 wording verbatim.
|
||
|
||
**What remains for this phase:** a test proving isolation end-to-end now that two plugins (`user`, `fonoteka`) both have real migration sets with several migrations each — `summer migrate:rollback --plugin=fonoteka` must roll back only Fonoteka's last migration (e.g. `create_wishlist_digest_queue`) and leave `user`'s history table and `fonoteka`'s earlier migrations untouched. No new `lagoon` code is expected; this is a verification task, sized as part of the unit-test plan (or a smoke test earlier), not new command-writing.
|
||
|
||
## Don't Hand-Roll
|
||
|
||
| Problem | Don't Build | Use Instead | Why |
|
||
|---------|-------------|-------------|-----|
|
||
| Many-to-many with extra pivot columns | Custom join-table query layer from scratch | GORM `SetupJoinTable` + explicit sync function for writes (Association mode has no pivot-column write path) | GORM's own relation loading (with `ORDER BY`) still works for reads once `SetupJoinTable` is registered; only writes need the explicit path |
|
||
| Thumbnail generation | Hand-written crop/scale math on `image.Image` | `disintegration/imaging` (`Fill`/`Fit`/`Resize`) | Exactly matches Winter's three thumb modes; pure Go, no cgo, single dependency |
|
||
| Rule-string validation | A bespoke mini-parser for Laravel rule strings | `go-playground/validator`'s `Var()` API with a small string of go-playground tags produced by translating the PHP string once per field | Already the named STACK.md decision; hand-rolling reinvents an established, well-tested validation engine |
|
||
| Encrypted column storage | Ad-hoc `crypto/aes` calls scattered per model | One `lagoon.Encrypted` `sql.Scanner`/`driver.Valuer` type, used by all 4 credential models | Consistent key derivation (HKDF from `app.key`), consistent versioned ciphertext format, one place to audit for the security review |
|
||
| File storage abstraction | Direct `os.WriteFile`/`os.ReadFile` calls per attachment site | `gocloud.dev/blob` (`fileblob` in prod/dev, `memblob` in tests) | Already the named D-15 decision; gives a clean local↔S3/GCS swap path later with zero call-site changes |
|
||
|
||
**Key insight:** every "don't hand-roll" item in this phase already has a named library decision in CONTEXT.md or STACK.md — the research task was to verify each library's actual API shape against the specific PHP behavior it must reproduce (pivot columns, thumb naming, rule grammar), not to discover new libraries.
|
||
|
||
## Common Pitfalls
|
||
|
||
(See `.planning/research/PITFALLS.md` Pitfalls 3, 4, 5, 6, 7, 11, 12, 13 — all directly apply to this phase and are not re-derived here. Two phase-specific additions from this session's research:)
|
||
|
||
### Pitfall: go-playground/validator's `min`/`max` tags don't parse a custom string-backed Money type
|
||
**What goes wrong:** Applying `validate:"omitempty,min=0,max=999999.9999"` directly to a `MoneyString` (or similarly named string-backed) type either fails to compile validation against it meaningfully or silently no-ops, because the built-in `hasMinOf`/`hasMaxOf` functions dispatch on Go kind (numeric vs. string-length), and a named string type validates as a **string length** bound (`min=0` chars, `max=999999.9999` — not even a valid integer length), not a numeric value bound.
|
||
**Why it happens:** The PHP `numeric|min:0|max:999999.9999` rule operates on the raw numeric value; the Go port's "never float64 in the JSON path" decision (correctly) uses a string-backed type for the persisted/serialized representation, creating a type mismatch with validator's kind-based dispatch.
|
||
**How to avoid:** Validate the parsed `float64` (or a decimal intermediate) against `min`/`max` **before** constructing the `MoneyString`, or register a custom validation function (`validate.RegisterValidation("money_range", func(fl validator.FieldLevel) bool {...})`) that parses the string first.
|
||
**Phase to address:** Data layer (this phase) — the Money cast's validation wiring, not the cast itself.
|
||
|
||
### Pitfall: `oneof` requires single-quoting for any allowed value containing a space, and Album's own `FORMATS` constant has one (`EP 7"`)
|
||
**What goes wrong:** A naive `oneof=LP 2LP CD 2CD MC Box EP 7"` tag string splits on whitespace and treats `EP`, `7"` as two separate allowed values, neither of which matches the real value `EP 7"`.
|
||
**Why it happens:** Copying the PHP `Rule::in([...])` array items directly into a space-joined `oneof=` tag without noticing one value itself contains a space.
|
||
**How to avoid:** Wrap any oneof value containing whitespace in single quotes: `oneof='EP 7"' LP 2LP CD 2CD MC Box` (verified against the validator library's own `parseOneOfParam2` source, which strips single-quote delimiters after splitting — the literal `"` character inside the quoted value is not a delimiter and passes through safely).
|
||
**Phase to address:** Data layer (this phase) — `Album.Rules()["format"]`.
|
||
|
||
## Code Examples
|
||
|
||
### Winter `File` thumb filename + partition rule (D-17, verified against `vendor/winter/storm/src/Database/Attach/File.php`)
|
||
```php
|
||
// Source: winter/storm File.php:634-646 (getThumbFilename) and :1046-1049 (getPartitionDirectory)
|
||
// Thumb filename: thumb_<id>_<width>_<height>_<offsetX>_<offsetY>_<mode>.<ext>
|
||
// e.g. thumb_42_200_200_0_0_crop.jpg — matches SerializesFonoteka's getThumb(200, 200, ['mode' => 'crop'])
|
||
implode('_', ['thumb', (string) $this->id, (string) $width, (string) $height,
|
||
(string) $options['offset'][0], (string) $options['offset'][1],
|
||
(string) $options['mode'] . '.' . (string) $options['extension']]);
|
||
|
||
// Partition directory: first 9 chars of disk_name, split into 3 groups of 3, joined by '/'
|
||
implode('/', array_slice(str_split($this->disk_name, 3), 0, 3)) . '/';
|
||
// e.g. disk_name "abc123xyz.jpg" -> partition "abc/123/xyz/"
|
||
// Full public path: getPublicPath() . partitionDirectory . filename
|
||
// Full disk path: getStorageDirectory() . partitionDirectory . filename (getStorageDirectory: "public/" or "protected/")
|
||
```
|
||
**Go port:** a `Thumb(w, h, mode) string` method on the framework `File` struct reproduces both functions verbatim (string formatting only, no external dependency needed for the naming itself — only the actual pixel resize needs `disintegration/imaging`).
|
||
|
||
### `system_files` verified column set (D-14, `[VERIFIED: live Postgres introspection]`)
|
||
```sql
|
||
-- id, disk_name, file_name, file_size, content_type, title, description,
|
||
-- field, attachment_id, attachment_type, is_public, sort_order,
|
||
-- created_at, updated_at, metadata
|
||
-- NOTE: attachment_id is VARCHAR(255), not integer — Winter's morphMany
|
||
-- stores the owner's primary key as a STRING in this column, not an int FK.
|
||
```
|
||
|
||
### `disintegration/imaging` thumbnail generation matching Winter's modes
|
||
```go
|
||
// Source: pkg.go.dev/github.com/disintegration/imaging (verified function signatures)
|
||
import "github.com/disintegration/imaging"
|
||
|
||
func makeThumb(src image.Image, w, h int, mode string) *image.NRGBA {
|
||
switch mode {
|
||
case "crop":
|
||
return imaging.Fill(src, w, h, imaging.Center, imaging.Lanczos) // Winter mode=crop
|
||
case "exact":
|
||
return imaging.Resize(src, w, h, imaging.Lanczos) // Winter mode=exact (stretch)
|
||
default: // "auto"
|
||
return imaging.Fit(src, w, h, imaging.Lanczos) // Winter mode=auto (contain)
|
||
}
|
||
}
|
||
```
|
||
|
||
## State of the Art
|
||
|
||
| Old Approach | Current Approach | When Changed | Impact |
|
||
|--------------|------------------|---------------|--------|
|
||
| ARCHITECTURE.md/PITFALLS.md prose mentioning "goose" as the migration tool | `gormigrate` | Recorded as authoritative in STATE.md's Phase 3 decision log ("treat gormigrate as authoritative when phases 3 and 5 are planned") | This phase's migrations must use `gormigrate`'s `[]*gormigrate.Migration{ID, Migrate, Rollback}` shape (already the pattern in `fonoteka.go/plugins/golem15/fonoteka/migrations.go`), not goose files |
|
||
|
||
**No other "deprecated/outdated" findings** — this phase ports a stable, historical PHP codebase onto a current, already-decided Go stack; there is no legacy-Go-approach angle here.
|
||
|
||
## Assumptions Log
|
||
|
||
| # | Claim | Section | Risk if Wrong |
|
||
|---|-------|---------|---------------|
|
||
| A1 | `widen_users` can legitimately be empty/omitted for this phase's own success criteria (no Fonoteka model touches non-`id` user columns) | Squashed Migration List, `widen_users` finding | Low — if wrong, the planner adds columns later in a follow-up ALTER; nothing else in this phase depends on `widen_users` having content, since only `id` is an FK target |
|
||
| A2 | `golem15_user_organisations` migration belongs in the `user` plugin's set, not `fonoteka`'s | Squashed Migration List | Low-medium — if the planner instead puts it in `fonoteka` with a defensive existence-guard (mirroring PHP's own `Schema::hasTable` guard), functionality is identical; only the migration-ownership convention changes. Flagged because it affects which plugin's migration file the planner writes it into. |
|
||
| A3 | `disintegration/imaging` (not actively committed to since 2023, though not archived) is an acceptable pick despite low recent commit activity, because the feature set needed (Fill/Fit/Resize) is complete and stable | Standard Stack, D-18 | Low — the library does exactly what's needed with zero cgo and one clean transitive dependency; if a security issue surfaced in it later, the maintained `kovidgoyal/imaging` fork is a documented drop-in alternative |
|
||
| A4 | The `styles`/`album_styles` tables need a fresh Go migration (not folded from an existing Phase-3 artifact) since Phase 3 only created genres/collections/albums/collection_editors/user_collection_contexts | Squashed Migration List | Low — directly confirmed by reading `fonoteka.go/plugins/golem15/fonoteka/migrations.go`'s actual Phase-3 migration content in this session |
|
||
|
||
**If this table is empty:** N/A — see above; all four assumptions are low-risk implementation-convention calls, not unverified factual claims about the PHP source or the target Go libraries (which were all verified live in this session).
|
||
|
||
## Open Questions (RESOLVED)
|
||
|
||
1. **RESOLVED (user decision, 2026-09-18): `widen_users` is deferred to Phase 7 — no `widen_users` migration ships in Phase 5.** (Original question: does `widen_users` need `organisation_id`/`organisation_role` now, or does Phase 7 add them?)
|
||
- What we know: no Fonoteka model in this phase reads them; the columns exist in the real PHP `users` table and are cheap/additive.
|
||
- What's unclear: whether the planner wants to front-load this to avoid a second `users` ALTER in Phase 7, or keep Phase 5 strictly scoped to what this phase's own success criteria need.
|
||
- Recommendation: default to NOT adding them in Phase 5 (keeps the phase's `widen_users` migration honestly empty/minimal, matching "plus the golem15.user tables they depend on" in the phase boundary text); let Phase 7 (AUTH-02, Organisations with roles) own its own widen migration, since that's where the columns' actual behavior lands.
|
||
|
||
2. **RESOLVED (user decision, 2026-09-18): dedicated typed table `golem15_fonoteka_settings`, not Winter's generic `system_settings` mechanism.** (Original question: where does the `Settings` model's storage live, given it has no dedicated migration in the 38 PHP files?)
|
||
- What we know: only one field, `search_use_typesense` (boolean), is used; Winter's `SettingsModel` behavior serializes the whole settings array into one `system_settings.value` text column keyed by `item = 'golem15_fonoteka_settings'`.
|
||
- What's unclear: whether to port Winter's generic serialized-value mechanism (more PHP-parity-faithful but adds a PHP-`serialize()`-format decoder nobody else needs) or give `Settings` its own tiny dedicated table with a typed `search_use_typesense BOOLEAN` column (simpler, matches this project's "matching final schema" spirit at the field level even though the storage *mechanism* differs from PHP).
|
||
- Recommendation: dedicated table (`golem15_fonoteka_settings`, singleton row, typed boolean column) — Phase 9's ADMIN-05 needs a settings screen bound through "the same schema pipeline" regardless of storage shape, and a typed column is strictly easier for both this phase and Phase 9 than porting Winter's generic key-value behavior for a single flag. This is a recommendation, not a locked decision — confirm with the user/planner since it's a deliberate departure from PHP's storage mechanism (not from PHP's *data*, which is preserved).
|
||
|
||
3. **RESOLVED: no FK constraints on `notifications`/`wishlist_subscriptions`/`wishlist_digest_queue`'s `user_id`/`collection_id` — matches PHP exactly, not fixed as an oversight.** (Original question: exact FK columns to declare for `notifications` and `wishlist_subscriptions`/`wishlist_digest_queue`'s `user_id`/`collection_id`.) PHP deliberately omits FKs on these (confirmed: no `->foreign()` calls in the live schema for `golem15_fonoteka_notifications`, `golem15_fonoteka_wishlist_subscriptions`, `golem15_fonoteka_wishlist_digest_queue`), per the migration comments ("this plugin's convention... is to avoid cross-table FKs that already cascade-delete via a different owning relation"). Recommendation: match PHP exactly — no FK constraints on these three tables' `user_id`/`collection_id` columns, even though every other table does declare them. This is already effectively decided by D-02's "match PHP's actual constraints" framing; flagged here only so the planner doesn't "fix" this as an oversight.
|
||
|
||
## Environment Availability
|
||
|
||
| Dependency | Required By | Available | Version | Fallback |
|
||
|------------|------------|-----------|---------|----------|
|
||
| Docker | D-02 schema-diff test (testcontainers), fuzz tests | Yes | Confirmed working in this session | — |
|
||
| PostgreSQL (via testcontainers `postgres:16-alpine`) | All model/migration tests | Yes | 16.15, confirmed pl-PL ICU locale support already in use (`lagoon/postgres_test.go`) | — |
|
||
| PHP + Composer + `pdo_pgsql` | Producing the D-02 golden snapshot (one-time, offline) | Yes | PHP 8.x (Winter/Laravel 9.52.21), `pdo_pgsql` extension loaded | Not needed at CI time — snapshot is generated once and committed; CI never needs PHP |
|
||
| `disintegration/imaging` Go module | Thumbnail generation (D-15/D-17/D-18) | Yes (via `go get`) | v1.6.2, confirmed on `proxy.golang.org` | — |
|
||
|
||
**Missing dependencies with no fallback:** none.
|
||
**Missing dependencies with fallback:** none — everything needed for this phase is either already present or trivially `go get`-able.
|
||
|
||
## Validation Architecture
|
||
|
||
### Test Framework
|
||
| Property | Value |
|
||
|----------|-------|
|
||
| Framework | Go stdlib `testing` + `testify` (assert/require) + `testcontainers-go`/`modules/postgres` v0.44.0 |
|
||
| Config file | none — plain `func TestX(t *testing.T)`, `TestMain` pattern per `lagoon/postgres_test.go` (pl-PL ICU locale container) |
|
||
| Quick run command | `go test ./... -short` (skips testcontainers-backed tests per the existing `testShort()` convention) |
|
||
| Full suite command | `go test ./...` (starts real Postgres via testcontainers) |
|
||
|
||
### Phase Requirements → Test Map
|
||
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||
|--------|----------|-----------|-------------------|-------------|
|
||
| DATA-03 | Cascading soft-delete (Collection→Albums) in one transaction | integration | `go test ./plugins/golem15/fonoteka/... -run TestCollectionBeforeDeleteCascadesAlbums` | ❌ Wave 0 |
|
||
| DATA-04 | 3+ artist album round-trips `sort_order`; CollectionEditor pivot columns round-trip | integration | `go test ./plugins/golem15/fonoteka/... -run TestAlbumArtistsOrderRoundTrip` | ❌ Wave 0 |
|
||
| DATA-05 | Rule-string → 422 map for Album/Collection/Artist/Style/Genre/Settings | unit + integration (unique:table needs DB) | `go test ./lagoon/... -run TestValidate` | ❌ Wave 0 |
|
||
| DATA-06 | Fillable fuzz test (unknown fields never persisted) | integration | `go test ./plugins/golem15/fonoteka/... -run TestAlbumWriteServiceFillFuzz -fuzz` | ❌ Wave 0 |
|
||
| DATA-07 | Money cast round-trips PHP ceiling/blank cases as fixed string; encrypted cast round-trips + hidden | unit + integration | `go test ./lagoon/... -run TestEncrypted`, `go test ./plugins/golem15/fonoteka/models/... -run TestMoneyString` | ❌ Wave 0 |
|
||
| DATA-08 | Thumb filename matches Winter's exact rule; file delete-then-recreate on soft-deleted owner keeps blob | unit + integration | `go test ./lagoon/attach/... -run TestThumbFilename`, `-run TestFileLifecycle` | ❌ Wave 0 |
|
||
| DATA-09 | Schema diff: Go migrations up == PHP schema snapshot | integration | `go test ./parity/... -run TestSchemaMatchesPHPSnapshot` | ❌ Wave 0 (snapshot file itself also ❌) |
|
||
| DATA-10 | Pagination envelope shape (`{data, meta{...}}`, no `links`) | unit | `go test ./lagoon/... -run TestPaginate` | ❌ Wave 0 |
|
||
| DATA-11 | Cross-plugin callback fires without editing owning model; companion migration | integration | `go test ./plugins/... -run TestCrossPluginCallback` | ❌ Wave 0 |
|
||
| CLI-03 | `migrate:rollback --plugin=fonoteka` isolates to Fonoteka's own history table | integration | `go test ./cmd/summer/... -run TestRollbackScopedToPlugin` | ❌ Wave 0 |
|
||
|
||
### Sampling Rate
|
||
- **Per task commit:** `go test ./... -short`
|
||
- **Per wave merge:** `go test ./...` (full, real Postgres via testcontainers)
|
||
- **Phase gate:** Full suite green before `/gsd:verify-work`, plus the D-02 schema-diff test green
|
||
|
||
### Wave 0 Gaps
|
||
- [ ] `fonoteka.go/parity/testdata/php_schema_snapshot.sql` — the committed D-02 golden snapshot (generate once via the reproducible method above)
|
||
- [ ] `lagoon/fill_test.go`, `lagoon/validate_test.go`, `lagoon/encrypted_test.go`, `lagoon/paginate_test.go` — framework-level unit tests for the new `lagoon` primitives this phase adds
|
||
- [ ] `lagoon/attach/` package itself does not exist yet — this phase creates it (File model, blob wiring, Thumb())
|
||
- [ ] Existing test infra (`lagoon/postgres_test.go` TestMain pattern, `fonoteka.go/parity` TestMain) is reused, not rebuilt
|
||
|
||
## Security Domain
|
||
|
||
### Applicable ASVS Categories
|
||
|
||
| ASVS Category | Applies | Standard Control |
|
||
|---------------|---------|-------------------|
|
||
| V2 Authentication | No | Out of scope — Phase 7 |
|
||
| V3 Session Management | No | Out of scope — Phase 7/8 |
|
||
| V4 Access Control | Partial | Mass-assignment fillable allow-list (D-05/D-06) is an access-control-adjacent control at the model layer; full authorization scoping is Phase 6/12 |
|
||
| V5 Input Validation | Yes | go-playground/validator rule translation (D-09), fillable allow-list enforcement (D-05/D-06) |
|
||
| V6 Cryptography (STORAGE) | Yes | AES-256-GCM via `lagoon.Encrypted`, key derived via HKDF from `app.key` (D-10..D-13) — never hand-rolled AES, matching the PHP source's own explicit "Hand-rolled AES is forbidden in this repository" comment on `UserDiscogsCredential`/`OrgDiscogsCredential` |
|
||
|
||
### Known Threat Patterns for this stack
|
||
|
||
| Pattern | STRIDE | Standard Mitigation |
|
||
|---------|--------|----------------------|
|
||
| Mass assignment via unknown/server-owned fields (`collection_id`, `market_price_source` on Album; `public_token`/`public_enabled`/`kind` on Collection) | Tampering / Elevation of Privilege | Two-layer fillable: model `Fillable()` backstop + narrower per-service allow-list (`AlbumWriteService.FILL_FIELDS`), fuzz-tested (D-07) |
|
||
| Accidental serialization of `token_hash`/`code_hash`/`client_secret_hash`/encrypted credential columns | Information Disclosure | `json:"-"` struct tag as a hard backstop + `Hidden()` method for a lagoon-wide "marshal every model, assert hidden absent" test (D-08); credentials additionally never reach plaintext except through explicit `.Reveal()` |
|
||
| Encrypted column readable by an old/rotated key holder after rotation | Information Disclosure (if mishandled) / Denial of Service (if not) | Versioned ciphertext format (key-id prefix) + `app.previous_keys` decrypt-only fallback (D-12), same pattern as Laravel's `APP_PREVIOUS_KEYS` |
|
||
| Soft-delete + unique constraint bypass allowing a "deleted" row's unique key to block a legitimate new row, or vice versa a delete-then-recreate silently succeeding when PHP would have rejected it (data integrity, not directly a STRIDE category but a parity/correctness risk with security-adjacent stakes for the `public_token` case — token reuse across a delete boundary) | Tampering (data integrity) | Match PHP's actual constraint shape exactly (plain unique, no partial index) per the audit above — do not "improve" this with a partial index PHP doesn't have, since that would be an undocumented behavior change on a token-bearing column |
|
||
| `unique:table` validation check racing a concurrent write (TOCTOU) | Tampering | Not newly introduced by this phase (PHP has the same race); note for the planner that the DB-level unique constraint, not the pre-save validator check, is the actual integrity guarantee — the validator check is UX (early 422), not the security boundary |
|
||
|
||
## Sources
|
||
|
||
### Primary (HIGH confidence)
|
||
- **Live Postgres 16 schema introspection** (this session): ran the actual PHP `winter:up` migration command via `php artisan winter:up` against a scratch `postgres:16-alpine` container, then `\d <table>` / `pg_dump --schema-only` for all 25 Fonoteka tables, `system_files`, `users`, `golem15_user_organisations`. This is the primary source for the Full Per-Model Inventory's column/index/FK facts and for answering D-02.
|
||
- Direct reads of all 25 PHP model files (`/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/models/*.php`)
|
||
- Direct reads of all 38 PHP migration files (`updates/v*/*.php`) plus `updates/version.yaml`
|
||
- Direct reads of `classes/casts/MarketPriceCast.php`, `classes/AlbumWriteService.php`, `traits/SerializesFonoteka.php`
|
||
- Direct read of `vendor/winter/storm/src/Database/Attach/File.php` (thumb naming, partition directory, storage directory logic — lines 440-1050)
|
||
- Direct read of `config/cms.php` (storage.uploads config shape)
|
||
- Direct read of `modules/system/database/migrations/2013_10_01_000002_Db_System_Files.php` and the 2025 metadata-column migration
|
||
- `gorm.io/docs/many_to_many.html` (fetched live) — `SetupJoinTable` exact API
|
||
- `github.com/go-playground/validator` `doc.go` and `baked_in.go` (fetched live) — confirmed no `between` tag, `oneof` single-quote parsing, `min`/`max`/`numeric`/`number`/`omitempty`/`omitnil` semantics
|
||
- `proxy.golang.org` `.info` endpoints (fetched live) — exact version/date confirmation for `disintegration/imaging`, `go-playground/validator/v10`, `gocloud.dev`, `golang.org/x/image`
|
||
- `pkg.go.dev/gocloud.dev/blob/fileblob` (fetched live) — `OpenBucket` signature, `Options` fields, subdirectory-key handling
|
||
- `laravel.com/docs/11.x/encryption` (fetched live) — confirms AES-256-CBC + HMAC MAC format for the Phase 15 decrypt-only helper (D-10)
|
||
- Existing `summercms.go` source: `lagoon/migrations.go`, `lagoon/commands.go`, `lagoon/postgres_test.go`, `pact/capabilities.go` — CLI-03's already-implemented status, existing test patterns
|
||
- Existing `fonoteka.go` source: `plugins/golem15/fonoteka/{migrations.go,genre.go,plugin.go}`, `plugins/golem15/user/{migrations.go,user.go,plugin.go}`, `parity/migrate_test.go` — Phase 3 baseline to widen
|
||
|
||
### Secondary (MEDIUM confidence)
|
||
- `.planning/research/ARCHITECTURE.md`, `PITFALLS.md`, `STACK.md` — prior-phase research, cross-checked against this session's live findings (no contradictions found; `gormigrate`-vs-`goose` staleness explicitly flagged and resolved per STATE.md)
|
||
|
||
### Tertiary (LOW confidence)
|
||
- `pkg.go.dev/github.com/disintegration/imaging` function-signature summary (WebFetch-summarized, not raw source read) — the function signatures (`Fill`, `Fit`, `Resize`, `Thumbnail`) and filter constant list are standard/well-known for this library and were cross-checked against the confirmed go.mod (only `golang.org/x/image` as a dependency, implying pure-Go/no-cgo) but were not read from raw source in this session
|
||
|
||
## Metadata
|
||
|
||
**Confidence breakdown:**
|
||
- Standard stack: HIGH — all versions verified live against `proxy.golang.org`; `disintegration/imaging` verified not archived, dependency graph confirmed minimal
|
||
- Architecture (schema, models, migrations): HIGH — the entire schema was verified by actually running the PHP migrations against a real Postgres instance, not inferred from reading migration files
|
||
- Pitfalls: HIGH — Pitfalls 3/4/5/6/7/11/12/13 are already-established project research; the two new pitfalls (validator min/max on Money type, oneof quoting) were verified against the validator library's own source
|
||
- Security domain: HIGH — directly grounded in PHP source comments that explicitly name the security invariants ("Hand-rolled AES is forbidden in this repository", "$hidden... must never be returned to the SPA")
|
||
|
||
**Research date:** 2026-09-18
|
||
**Valid until:** 30 days for the Go library version pins (stable ecosystem); the live-verified PHP schema facts do not expire unless the PHP source changes (not expected mid-port)
|