195 lines
8.0 KiB
Markdown
195 lines
8.0 KiB
Markdown
# Phase 5: Data layer full fidelity - Discussion Log
|
|
|
|
> **Audit trail only.** Do not use as input to planning, research, or execution agents.
|
|
> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
|
|
|
|
**Date:** 2026-09-18
|
|
**Phase:** 5-Data layer full fidelity
|
|
**Areas discussed:** Todo folding, Migration granularity, Model DX (fillable, hidden, rules), Encrypted cast compatibility, Attachments depth (DATA-08)
|
|
|
|
---
|
|
|
|
## Todo folding
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Fold in as first task | Run the models-leaf probe on keios.eu, jz and pxpx plugins at the start of Phase 5; amend the note if an "other" edge appears | ✓ |
|
|
| Keep it separate | Leave as a pending todo; apply the rule on Fonoteka evidence only | |
|
|
|
|
**User's choice:** Fold in as first task
|
|
|
|
---
|
|
|
|
## Migration granularity
|
|
|
|
Context raised: P3 D-16 (squash per table) conflicts with roadmap criterion 1 ("all 27 migrations run up and down individually"); `version.yaml` actually lists 38 files across 26 versions.
|
|
|
|
### Relation to PHP history
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Squash per table | Keep P3 D-16; one create migration per final-state table, commented with folded PHP files; reword criterion 1 | ✓ |
|
|
| Mirror PHP one-to-one | One Go migration per PHP file including renames and the items-to-albums flatten | |
|
|
| Squash per PHP version | One Go migration per version.yaml entry with a final-state effect | |
|
|
|
|
### Proving schema equality
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Schema-diff test vs PHP dump | Committed normalized snapshot of PHP's final Postgres schema; testcontainers test diffs information_schema with a documented allow-list | ✓ |
|
|
| Hand-audited checklist | Reviewer reads migration comments against PHP | |
|
|
| You decide | Researcher picks after checking PHP migrations on Postgres | |
|
|
|
|
### Widening the Phase 3 tables
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| One widen migration per table | widen_albums, widen_collections, widen_users after the Phase 3 pair | ✓ |
|
|
| Single "phase 5 widen" migration | One ALTER migration for all three | |
|
|
| You decide | Planner orders by FK needs | |
|
|
|
|
### Data seeds
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Idempotent data migrations | Same as genres seed (P3 D-18) | ✓ |
|
|
| Separate seed command | Schema-only migrations plus `fonoteka:seed` | |
|
|
|
|
---
|
|
|
|
## Model DX: fillable, hidden, rules
|
|
|
|
Context raised: PHP API never serializes models via `toArray()` (`SerializesFonoteka` builds payloads); `AlbumWriteService::FILL_FIELDS` is a strict subset of `Album::$fillable`.
|
|
|
|
### Declaring fillable
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Declared list + lagoon.Fill | `Fillable() []string` copied from PHP; `lagoon.Fill` intersects requested and fillable names; services pass narrower lists | ✓ |
|
|
| Struct tags | `summer:"fillable"` tags read by reflection | |
|
|
| Hand-written DTO copy only | No framework concept; DTO shape is the only boundary | |
|
|
|
|
### Unknown / non-fillable keys
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Silently ignored, as PHP | Dropped, never persisted; logged once in non-production | ✓ |
|
|
| Reject with 422 | Stricter than PHP; changes the contract | |
|
|
|
|
### Proving criterion 3 without write routes
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Service-level fuzz now | Port write services' fill boundary and fuzz against Postgres; HTTP fuzz in Phase 12 | ✓ |
|
|
| One real write route now | Pull a write route forward as the DTO reference | |
|
|
| Framework-only test | Fuzz lagoon.Fill on a fixture model only | |
|
|
|
|
### Hidden
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| json:"-" + explicit serializers | `json:"-"` on secrets, `Hidden()` list, marshal test, named Reveal override | ✓ |
|
|
| Generic lagoon.Serialize(model, opts) | Reflection serializer with WithVisible option | |
|
|
|
|
### Rules
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Rules() map on model, validated on save | PHP strings verbatim; lagoon translates to validator; Laravel-shaped 422 | ✓ |
|
|
| validate struct tags | Hand-translated go-playground tags per field | |
|
|
| You decide | Researcher evaluates grammar coverage first | |
|
|
|
|
---
|
|
|
|
## Encrypted cast compatibility
|
|
|
|
### Existing Laravel ciphertext
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| GCM only; re-encrypt at import | Cast handles only AES-GCM; ship a Laravel decrypt helper for the Phase 15 import | ✓ |
|
|
| Cast reads both, writes GCM | CBC decrypt on the live read path, lazy re-encrypt | |
|
|
| No migration of secrets | Users re-enter keys after cutover | |
|
|
|
|
### Key source
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| app.key, fail boot if missing | 32-byte base64, HKDF-derived column key, `summer key:generate` | ✓ |
|
|
| Dedicated database.encryption_key | Separate secret for column encryption | |
|
|
|
|
### Rotation
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Versioned format + previous keys | Key-id prefix; `app.previous_keys` decrypt-only | ✓ |
|
|
| Single key, no version prefix | Simplest; rotation needs a format change later | |
|
|
|
|
### Go type
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| lagoon.Encrypted type | Scanner/Valuer; redacting MarshalJSON/String; explicit Reveal() | ✓ |
|
|
| Plain string + serializer tag | `gorm:"serializer:encrypted"` on a string | |
|
|
|
|
---
|
|
|
|
## Attachments depth (DATA-08)
|
|
|
|
### Table
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Winter's system_files shape | Same table and columns, PHP class strings as morph names, framework-owned | ✓ |
|
|
| New clean table | Go-native `summer_files` | |
|
|
|
|
### Storage depth
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Table, relations and path/URL math only | Recommended cut; blob and thumbs later | |
|
|
| Full blob storage now | gocloud.dev/blob in this phase, thumbs later | |
|
|
| Everything incl. thumbnails | Blob storage plus 200x200 crop thumbs | ✓ |
|
|
|
|
**Notes:** User picked the largest scope over the recommendation. HTTP upload endpoint remains Phase 12.
|
|
|
|
### Thumb shape
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Same name and path, lazy on first call | Winter's thumb filename next to the original, generated when missing | ✓ |
|
|
| Same name, eager at attach time | Generate declared sizes on attach | |
|
|
| You decide | Researcher confirms Winter's rules first | |
|
|
|
|
### Image library
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| stdlib image + x/image/draw | No cgo, quasi-stdlib | |
|
|
| disintegration/imaging | Convenient, unmaintained since 2020 | |
|
|
| Researcher picks | Maintained pure-Go option recorded in RESEARCH.md; no cgo | ✓ |
|
|
|
|
### Bucket and serving
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| fileblob at storage/app, config-driven URL, Go static handler | Mirrors cms.php storage.uploads; memblob in tests | ✓ |
|
|
| Same, no Go static handler | Reverse proxy serves files | |
|
|
|
|
### Delete behavior
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| As Winter | Soft delete keeps files; force delete removes rows in-tx and blobs after commit | ✓ |
|
|
| Never delete blobs automatically | Prune command later | |
|
|
|
|
---
|
|
|
|
## Claude's Discretion
|
|
|
|
Hook naming and base model embed; pivot model shape; money cast type; jsonable cast shape; pagination helper API; callback-registry API and the cross-plugin extension demo pair; soft-delete + unique index strategy; which Serialize* functions are ported now; CLI-03 details; plan count and split.
|
|
|
|
## Deferred Ideas
|
|
|
|
HTTP-level DTO fuzz (Phase 12/13); upload endpoint, manual cover URL, Discogs cover import (Phase 12/14); re-encrypt command; cutover import of secrets and files (Phase 15); S3/GCS bucket; Typesense and Centrifugo hooks on Album (Phase 11); roadmap wording fixes ("27 migrations", criterion 3 endpoint clause) at plan time.
|