golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c97cc094d5de5d624799e5ac66bcb2c235c19e0d
summercms/.planning/phases/12-p-ytarium-api-collections-and-albums/12-04-SUMMARY.md

20 KiB
Raw Blame History

phase, plan, subsystem, tags, requires, provides, affects, actuals, plan_head_before, plan_head_after, tech-stack, key-files, key-decisions, patterns-established, requirements-completed, coverage, duration, completed, status
phase plan subsystem tags requires provides affects actuals plan_head_before plan_head_after tech-stack key-files key-decisions patterns-established requirements-completed coverage duration completed status
12-p-ytarium-api-collections-and-albums 04 api
fonoteka
albums
search
typesense
beachcomber
fetchguard
lighthouse
notifications
uploads
parity
tide
phase provides
12-p-ytarium-api-collections-and-albums 12-01: lagoon.ValidateRequest, beachcomber.SearchPage/QueryByWeights, attach URLs and webp, tide multipart; 12-02: Resolve/AlbumsAccessibleBy/requestScope/decodeInput/phpInt/laravelBoolean/SerializePhoto, the fonoteka seed hook; 12-03: WriteNotification, writeResolveError
phase provides
11-jobs-realtime-and-search-infrastructure lighthouse WithoutBroadcasting/Emit and the album binding, beachcomber after-commit sync, the broadcast goldens
classes CreateAlbum/UpdateAlbum (AlbumWriteService create/apply), ResolveArtistInputs/ArtistDisplayFor/SyncAlbumArtists, ResolveStyleIDs/SyncAlbumStyles, ParseTracklistText, ParseAddedDate, FindDuplicateAlbum/NormalizeForMatch/NormalizeBarcode, SyncCompletion/MissingTags/CompletenessPercent
classes NotifyAlbumAdded plus a GORM created hook (album_added for owner and editors minus the actor)
classes AlbumDTO/SerializeAlbum/SerializeAlbums/LoadAlbumDTO, ArtistDTO/ArtistAggregateDTO/GenreDTO/StyleDTO serializers, AlbumBroadcastPayload/AlbumChannels/EmitAlbumEvent
classes IsAllowedImage, CoverImporter/ImportAlbumCovers, ManualCoverFetcher/ManualCoverMessage, StoreAlbumPhoto
classes AlbumStats, AlbumValue/FormatValueTotal, MissingAlbums, AlbumSyncDelta/AlbumTombstones/EncodeSyncCursor, SearchAlbums (Scout-exact recount)
models.Slug/ASCII (Laravel Str::slug/Str::ascii), models.MarketCurrencyFunc wired to classes.MarketCurrency
36 album and lookup routes ported on both auth groups (99 ported routes in the corpus)
config keys golem15.fonoteka.discogs.{max_covers,cover_max_bytes,cover_timeout_seconds,cover_host_suffix} and golem15.fonoteka.covers.{manual_url_max_bytes,manual_url_timeout_seconds}
parity album state (PARITY_CASE=albums), 169 recorded album cases, the nuxt-albums flow, all four broadcast goldens asserted
12-05
13
14
tokens tasks commits
126088 3 5
fonoteka.go f88c01e787ecb541c91aee0638c2c21b36d62e9a; summercms.go 18ead668ec fonoteka.go 3b8304ea8cf6fe06af0972066e40e36e3ef2fe9a; summercms.go 44526f0bca
added patterns
Album writes run under WithoutBroadcasting[models.Album] in their own transaction; covers import after commit; the handler emits exactly one event through EmitAlbumEvent
A GORM callback handle is used only through a NewDB session (cleanSession): WithContext keeps the triggering write's bind variables
Parity album state with fixed ids and timestamps on both sides so cursors, orderings and sync versions are deterministic
Recorded query strings are restored from the spec after recording (the recorder scrubs short numbers as ids)
created modified
../fonoteka.go/plugins/golem15/fonoteka/classes/php_values.go
../fonoteka.go/plugins/golem15/fonoteka/classes/style_resolver.go
../fonoteka.go/plugins/golem15/fonoteka/classes/tracklist_text_parser.go
../fonoteka.go/plugins/golem15/fonoteka/classes/added_date_parser.go
../fonoteka.go/plugins/golem15/fonoteka/classes/duplicate_matcher.go
../fonoteka.go/plugins/golem15/fonoteka/classes/completeness.go
../fonoteka.go/plugins/golem15/fonoteka/classes/serialize_album.go
../fonoteka.go/plugins/golem15/fonoteka/classes/album_queries.go
../fonoteka.go/plugins/golem15/fonoteka/classes/album_broadcast.go
../fonoteka.go/plugins/golem15/fonoteka/classes/image_guard.go
../fonoteka.go/plugins/golem15/fonoteka/classes/album_files.go
../fonoteka.go/plugins/golem15/fonoteka/classes/cover_importer.go
../fonoteka.go/plugins/golem15/fonoteka/classes/manual_cover_fetcher.go
../fonoteka.go/plugins/golem15/fonoteka/classes/album_stats.go
../fonoteka.go/plugins/golem15/fonoteka/classes/album_sync.go
../fonoteka.go/plugins/golem15/fonoteka/classes/album_search.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/albums_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/albums_bulk_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/album_photos_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/album_stats_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/album_sync_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/album_search_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/ratings_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/lookups_controller.go
../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go
../fonoteka.go/parity/fixtures/nuxt/nuxt-albums.yaml
../fonoteka.go/parity/fixtures/routes/files/album-cover.webp
../fonoteka.go/plugins/golem15/fonoteka/classes/album_write_service.go
../fonoteka.go/plugins/golem15/fonoteka/classes/artist_resolver.go
../fonoteka.go/plugins/golem15/fonoteka/classes/notification_service.go
../fonoteka.go/plugins/golem15/fonoteka/classes/serialize.go
../fonoteka.go/plugins/golem15/fonoteka/classes/access.go
../fonoteka.go/plugins/golem15/fonoteka/models/slug.go
../fonoteka.go/plugins/golem15/fonoteka/models/album.go
../fonoteka.go/plugins/golem15/fonoteka/models/style.go
../fonoteka.go/plugins/golem15/fonoteka/realtime.go
../fonoteka.go/plugins/golem15/fonoteka/routes.go
../fonoteka.go/plugins/golem15/fonoteka/plugin.go
../fonoteka.go/plugins/golem15/fonoteka/config/config.yaml
../fonoteka.go/plugins/golem15/fonoteka/phase08_coverage_test.go
../fonoteka.go/parity/manifest.yaml
../fonoteka.go/parity/fonoteka_seed_test.go
../fonoteka.go/parity/fonoteka_reset.php
../fonoteka.go/parity/broadcast_goldens_test.go
../fonoteka.go/parity/fonoteka_flows_test.go
../fonoteka.go/parity/parity_test.go
../fonoteka.go/parity/parity_contract_test.go
../fonoteka.go/parity/README.md
../fonoteka.go/README.md
scripts/check-phase10.sh
scripts/check-phase10.1.sh
scripts/check-phase11.sh
Slugs and artist name keys now port Laravel Str::slug/Str::ascii (transliteration, punctuation removed): 'Czesław Niemen' is czeslaw-niemen and 'R&B' is rb as in PHP; non-Latin scripts are dropped, a documented gap
Album strings are trimmed on assignment (Winter trimStringAttributes), and an added date keeps its wall clock in UTC because Eloquent stores a Carbon without converting its zone
The search engine path re-gates engine ids with AlbumsAccessibleBy plus the active collection only, as Scout's query callback does; the plan's 'plus filters' would drop stale-index hits PHP still returns
Sync versions are compared at whole seconds (date_trunc) so the PHP cursor base64('<ISO>|<id>') round-trips exactly and since/checkpoint windows behave as on SQLite
Search name/artist sorts use the pl-x-icu collation with explicit SQLite NULL placement; GET albums and GET styles sort byte-wise (COLLATE "C") like the recorded SQLite BINARY order; id breaks every tie
albums/value sums are exact NUMERIC sums printed through a double with two decimals (correctly rounded), matching php -r sprintf('%.2f') including ties
A model-rule failure on update (an empty name) is PHP's ValidationException: the Winter 500 page, as recorded
The genres seed hook stays: its fixtures were recorded from the Phase 2 bootstrap state, not the fonoteka reset
Write-path handlers: resolveAlbumScope -> findAlbum (404 first) -> validate -> WithoutBroadcasting write -> covers after commit -> EmitAlbumEvent -> LoadAlbumDTO
Test seams for outbound fetches are injected functions (CoverImporter.Fetch, ManualCoverFetcher.Fetch, api.SetCoverImporterForTest), never fetchguard internals
API-02
id description requirement verification human_judgment
D1 POST albums and GET albums/{id} on both groups store and show albums like PHP (fill boundary, artists by id/name/Discogs id, styles, tracklist text, added dates, duplicate warning, Winter trimming) with one created event and household notifications API-02
kind ref status
e2e go -C ../fonoteka.go test ./parity -run TestParityCorpus (29 Task 1 cases) pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestAlbumStoreSingleCreatedEvent pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestAlbumAddedNotifiesHousehold pass
kind ref status
unit ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestAlbumWriteHelpersMatchPHP pass
false
id description requirement verification human_judgment
D2 All four album broadcast goldens (created, updated, deleted, bulk) are assertions driven through the real handlers; the gate scripts expect no skip API-02
kind ref status
integration go -C ../fonoteka.go test ./parity -run TestBroadcastGoldens pass
kind ref status
other scripts/check-phase10.sh --self-test; scripts/check-phase10.1.sh --self-test; scripts/check-phase11.sh --self-test pass
false
id description requirement verification human_judgment
D3 Album index, update, delete, ratings, photos (file and cover_url), bulk, stats, value, missing and sync answer PHP's bodies on their groups; covers import after commit through fetchguard; uploads pass the image guard API-02
kind ref status
e2e go -C ../fonoteka.go test ./parity -run TestParityCorpus (86 Task 2 cases) pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestCoverImportAfterCommit pass
kind ref status
unit ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestManualCoverReasons pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestAlbumPhotoUpload pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestBulkSingleSummaryEvent pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestRatingUpsertConcurrent pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestAlbumValueFormatting pass
false
id description requirement verification human_judgment
D4 Album search re-gates engine candidates and recounts the total in SQL (capped at 1000 ids), falls back to an escaped ILIKE search; artists, styles and genres lookups; the Nuxt albums journey replays API-02
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestAlbumSearchTypesenseRecount pass
kind ref status
integration ../fonoteka.go/plugins/golem15/fonoteka/albums_smoke_test.go#TestAlbumSearchSQLEscaping pass
kind ref status
e2e go -C ../fonoteka.go test ./parity -run TestFonotekaNuxtFlows/nuxt-albums pass
kind ref status
e2e go -C ../fonoteka.go test ./parity -run TestParityCorpus/coverage (99 ported); go run ./parity/check_corpus.go --require-recorded --check-secrets pass
false
82min 2026-10-02 complete

Phase 12 Plan 04: Albums, search and lookups Summary

The whole Albums surface of Płytarium on both auth groups: PHP's album write path (artists, styles, tracklist text, added dates, duplicates, completeness, Winter trimming) with one broadcast per write and household notifications, guarded cover imports and photo uploads, bulk, ratings, stats, value, missing and sync, Scout-exact album search with a SQL recount, the artists/styles/genres lookups, and the recorded Nuxt albums journey: 36 routes, 99 ported in the corpus.

Performance

  • Duration: 82 min
  • Started: 2026-10-02T11:19:49Z
  • Completed: 2026-10-02T12:41:55Z
  • Tasks: 3 (tracer verified end to end before expansion)
  • Files modified: 223 in fonoteka.go (code, tests, 169 fixtures, docs), 3 in summercms.go

Accomplishments

  • CreateAlbum/UpdateAlbum port AlbumWriteService create/apply: AlbumFillFields only (collection_id, market_price_source and server fields never persist), artists resolved by id, name_key or Discogs id with PHP's pivot sync (a repeated artist keeps its last position) and artist_display, styles by id, name or slug, tracklist_text, created_at through the AddedDateParser port, completeness stamping.
  • Every album insert notifies the household: owner and editors minus the actor get album_added, published after commit.
  • Store, update and bulk write under WithoutBroadcasting[models.Album] and emit exactly one created/updated event or one collection.bulk_updated; a failing bulk row rolls the batch back and emits nothing. All four broadcast goldens are assertions.
  • Covers: cover_urls are imported after the write commits through fetchguard AllowHosts (discogs.com), failures recorded in input order; the manual cover_url uses fetchguard PublicOnly with PHP's message per reason; uploads pass the ImageContentGuard port (webp accepted, polyglots refused).
  • Stats, value (exact sums printed like PHP sprintf), missing (tags, counters, dismissal) and sync (cursor, tombstones, checkpoint cap) answer PHP's bodies.
  • Search re-gates engine ids in SQL and recounts the total over at most 1000 engine ids (pages of 250); SQL-only sorts and rating filters skip the engine; engine errors fall back to an ILIKE search with %, _ and \ escaped.
  • 169 album cases recorded from the isolated PHP with a deterministic album state on both sides; the nuxt-albums flow replays; TestParityCorpus reports 99 ported routes and check_corpus --require-recorded --check-secrets is green.

Task Commits

  1. Task 1: album store/show, write path, notifications, created golden (tracer) - a01a750 (feat, fonoteka.go), 7ac6c05 (chore, summercms.go gate scripts)
  2. Task 2: edit, rate, photos, bulk, delete, sync, stats, value, missing - 4e6cfc6 (feat, fonoteka.go), 6649763 (chore, summercms.go gate scripts)
  3. Task 3: search, lookups, nuxt-albums flow - 3b8304e (feat, fonoteka.go)

Ledgers (git rev-list --count): fonoteka.go f88c01e..3b8304e, 3 commits; summercms.go 18ead66..44526f0, 3 commits, of which 2 are this plan's (44526f0 is the user's concurrent Phase 12.2 roadmap insertion).

Files Created/Modified

  • classes/* (16 new files): the write-path helpers, serializers, broadcast payload, image guard, covers, stats, sync and search
  • controllers/api/* (8 new files): the album, bulk, photo, rating, stats, sync, search and lookup handlers
  • models/slug.go, models/album.go, models/style.go: Str::slug/Str::ascii, configurable market currency, exported StyleSlug
  • routes.go, config/config.yaml, plugin.go, realtime.go: 36 routes, cover config keys, market currency and realtime wiring
  • parity/*: album state on both sides, manifest flips, 169 fixtures, the flow and its replay, golden tests through handlers, README recipe
  • scripts/check-phase10.sh, check-phase10.1.sh, check-phase11.sh: no pending golden skips; created and updated required

Decisions Made

See key-decisions in the frontmatter.

Deviations from Plan

Auto-fixed Issues

1. [Rule 1 - Bug] Slugs and name keys did not match Laravel

  • Found during: Task 1
  • Issue: slugify replaced punctuation with dashes and dropped non-ASCII letters ("R&B" became r-b, "Czesław" czes-aw); normalizeNameKey did no ASCII folding, so name_key deduplication differed from PHP.
  • Fix: models.Slug/models.ASCII port Str::slug/Str::ascii (checked against PHP output); name keys fold through ASCII.
  • Commit: a01a750

2. [Rule 1 - Bug] Notification writes broke inside GORM callbacks

  • Issue: WriteNotification used tx.WithContext, which keeps the statement's bind variables; called from a created callback, its INSERT numbered placeholders after the album insert's ("could not determine data type of parameter $1"). The same bug made the album broadcast payload fail.
  • Fix: both use a NewDB session (cleanSession).
  • Commit: a01a750

3. [Rule 1 - Bug] Embedded-struct scans panicked in the broadcast payload

  • Fix: explicit row structs for styles and artists in LoadAlbumEmbeds.
  • Commit: a01a750

4. [Rule 3 - Blocking] Deferred-scope route-table placeholders

  • Issue: Phase 8 placeholder subtests asserted no album route exists.
  • Fix: replaced with real checks (flat album routes on both surfaces, {id} constraints behaviourally, stats on both surfaces, sync and missing JWT only with one scope per token route) and a real port of OAuthRevocationTest's album-outside-the-pin 404; match/apply stay deferred to Phase 14.
  • Commits: a01a750, 4e6cfc6, 3b8304e

5. [Rule 3 - Blocking] Recording collisions in the parity tooling

  • Issue: var names with digits (id:album-2) were rewritten by the path scrubber, and short numbers in queries (per_page=2) were recorded as {{id:bob}}; album attachments of wiped albums leaked into later cases; validation messages under *_id/*_at keys hit the id/date masks.
  • Fix: digit-free var names, query strings restored from the spec after recording, all album attachments wiped on reset, normalize: disable on those message paths.
  • Commits: a01a750, 4e6cfc6

6. [Rule 1 - Bug] Broadcast goldens held raw album dates

  • Issue: created/updated were recorded before 12-01's album date masking, so they could not match the normalised Go publication.
  • Fix: the two album created_at/updated_at values are the {{datetime}} mask the normaliser produces; pending markers removed.
  • Commits: a01a750, 4e6cfc6

7. [Plan adjustment] Engine path filters

  • The plan said engine ids are re-gated "plus filters"; Scout's query callback applies only access and the active collection, so filters stay engine-side (see key decisions).

8. [Plan adjustment] Sync cases and flow timing

  • Sync recordings pin since/checkpoint to the seeded timestamps instead of using a live checkpoint, so sync versions and cursors are deterministic across PHP (second-precision SQLite) and Go.

Total deviations: 6 auto-fixed (4 bugs, 2 blocking) plus 2 plan adjustments. Impact on plan: All 36 routes, the flow and the goldens are delivered; every deviation follows the recorded PHP contract or keeps the shared corpus deterministic. No scope creep.

Issues Encountered

  • Commits were made on master in both repos, as in 12-01..12-03 (git.branching_strategy: none).
  • The user committed a Phase 12.2 roadmap insertion to summercms.go during the run (44526f0); it was left untouched.
  • Recording wrote uploads into the PHP checkout's storage/app/uploads/public; the 12 new files were removed and the tree matches its pre-run listing. The PHP server was stopped. Private vars stay in /tmp/summercms-parity/p1204 (mode 0600).

Known Stubs

None.

User Setup Required

None. The cover config keys default to PHP's values.

Next Phase Readiness

  • 12-05 can build the request-DTO fuzz on albumCreateRules/albumUpdateRules/albumBulkRules and the route table, the D-18 leak test on the scripted engine (scriptedEngine, setSearchSetting) and SearchAlbums, and coverage on the helpers' PHP truth tables.
  • Phase 13 adds the wishlist branch of NotifyAlbumAdded and the reservation key of AlbumDTO.

Phase: 12-p-ytarium-api-collections-and-albums Completed: 2026-10-02

Self-Check: PASSED

All created files listed in key-files exist on disk. Commits a01a750, 4e6cfc6 and 3b8304e exist in fonoteka.go and 7ac6c05 and 6649763 in summercms.go. Plan-level verification passed: go vet ./... and go test ./... -count=1 in fonoteka.go (root, parity, the fonoteka plugin and sm-user-plugin) and in summercms.go, TestParityCorpus with 99 ported and passing, TestFonotekaNuxtFlows (both flows), TestBroadcastGoldens (all four), the three gate self-tests, and check_corpus.go --require-recorded --check-secrets.

Reference in New Issue View Git Blame Copy Permalink