docs(12): research phase domain
This commit is contained in:
@@ -0,0 +1,710 @@
|
|||||||
|
# Phase 12: Płytarium API — Collections and Albums - Research
|
||||||
|
|
||||||
|
**Researched:** 2026-10-02
|
||||||
|
**Domain:** Go port of the PHP collection/album/household HTTP surface (parity-driven), with authorization, uploads, SSRF-guarded fetches, search re-gating and transactional mail jobs
|
||||||
|
**Confidence:** HIGH for the PHP contract and the existing Go seams (both read line by line this session); MEDIUM for the sizing and the plan split
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Phase 12 is mostly porting work, but it is much wider than CONTEXT.md's route list makes it look. The Go side has only thin stubs for most of what these routes need:
|
||||||
|
- `classes.SerializeAlbum` and `SerializeCollection` emit 8 and 4 keys.
|
||||||
|
- `classes.ResolveArtists` only checks numeric ids.
|
||||||
|
- `SaveAlbum` handles neither styles, tracklist text, dates, covers nor completeness.
|
||||||
|
- `ResolveActiveCollection` ignores personal-token pins, never provisions a collection and never runs the pending-invitation guard.
|
||||||
|
- There is no request-facing `AccessibleBy` that adds the token's `collection_ids` narrowing.
|
||||||
|
|
||||||
|
On top of the controllers, the PHP routes pull in about 15 helper classes that CONTEXT.md does not list: `AlbumCompletenessService`, `AlbumDuplicateMatcher`, `TracklistTextParser`, `AddedDateParser`, `OrgProvisioner`, `AiGate`, `DiscogsGate`, `DiscogsConfigResolver`, `OrgAccess`, `PolishOrder`, `DiscogsInputParser::normalizeBarcode`, `NotificationService::notifyAlbumAdded` (fired by an `eloquent.created` listener on every album create), plus the full `ArtistResolver` (name and Discogs-id resolution with `name_key` firstOrCreate). Each one changes a response body or the DB state a parity replay checks.
|
||||||
|
|
||||||
|
Four findings change locked decisions or roadmap wording and must reach the user before planning:
|
||||||
|
1. **D-16 point 3 is factually wrong for the installed Scout.** `laravel/scout` v10.25.0 recomputes `total` in SQL with the query callback applied. It re-fetches up to `min(found, max_total_results=1000)` ids from Typesense and counts the re-gated rows. So PHP's `meta.total` is the re-gated count, capped at 1000. It is not Typesense's `found`. A stale document still yields a short page, but the total is not inflated. Mirroring Scout exactly (D-16's own wording) means porting this two-step count.
|
||||||
|
2. **`me/context` does not return a channel name.** The recorded PHP body has no channel. The channel lives at `GET realtime/channels` → `{"data":{"collection":"collection:<id>"}}`. That route is pending in the manifest and the Nuxt app calls it. API-01's wording maps onto both routes, so `realtime/channels` should be ported here.
|
||||||
|
3. **Error bodies for HttpException routes are Winter HTML pages, not JSON.** This covers switch, household, invitations and accept. The existing accept fixture was recorded with the debug exception page (file paths and line numbers) and must be re-recorded under `APP_DEBUG=false`.
|
||||||
|
4. **Photo URLs don't match.** PHP serves `/storage/app/uploads/public/<partition>/<disk>` while Go's attach default is `/storage/uploads/<partition>/<disk>`. P5 D-16 required the PHP shape, so this phase (the first to emit photo URLs) must fix the config/layout.
|
||||||
|
|
||||||
|
Several framework gaps in `summercms.go` are blocking:
|
||||||
|
- `beachcomber` has no `found` count and no `query_by_weights`.
|
||||||
|
- `tide` can't record or replay multipart request bodies and doesn't mask random disk names in `url`/`thumb_url`.
|
||||||
|
- The validation layer lacks `array`, `*` wildcards, `url`, `date`, `after_or_equal`, `before_or_equal`, `exists`, `regex`, `string`, `image`, the size-type message variants (`max.string`/`max.array`/`max.numeric`/`max.file`) and Laravel's "stop after an implicit rule fails" semantics. The 422 bodies are recorded in Polish, e.g. `Pole albums jest wymagane.`.
|
||||||
|
- `attach` has no exported original-file URL helper.
|
||||||
|
|
||||||
|
**Primary recommendation:** Plan this as five plans:
|
||||||
|
1. Framework gaps (`summercms.go`).
|
||||||
|
2. Active context, collections and share.
|
||||||
|
3. Household, invitations and notifications, with the mail job.
|
||||||
|
4. Albums, search and lookups.
|
||||||
|
5. Unit tests, the fuzz and the security leak test.
|
||||||
|
|
||||||
|
Recordings go into each feature plan. First, take the four decision corrections above to the user.
|
||||||
|
|
||||||
|
## Architectural Responsibility Map
|
||||||
|
|
||||||
|
| Capability | Primary Tier | Secondary Tier | Rationale |
|
||||||
|
|------------|-------------|----------------|-----------|
|
||||||
|
| Tenant resolution (active collection, token pin, switch, provisioning) | API / Backend (`classes/active_collection.go`) | Database (row locks on users + contexts) | Server-resolved tenant. The client never selects one (C-01). |
|
||||||
|
| Access scoping (`accessibleBy` + token narrowing) | API / Backend (one GORM scope) | Database | One chokepoint shared by album, collection, search, stats and sync. |
|
||||||
|
| Request validation and 422 envelopes | API / Backend (request validator + phrasebook catalog) | — | Laravel semantics and messages are part of the contract. |
|
||||||
|
| Fill boundary and persistence | API / Backend (`AlbumWriteService`/`CollectionWriteService` ports) | Database | P5 D-05..D-07 allow-lists. The fuzz target. |
|
||||||
|
| Search candidate retrieval | External service (Typesense via `beachcomber`) | — | Candidate ids only, never authorization. |
|
||||||
|
| Search authorization and re-gate | Database (SQL `whereIn(ids)` + scopes) | API / Backend | Pitfall 15. |
|
||||||
|
| Photo/cover storage and thumbs | Storage (`gocloud.dev/blob` fileblob) | CDN/Static (static handler at the public prefix) | P5 D-14..D-19. |
|
||||||
|
| SSRF-guarded fetches (cover_urls, cover_url) | API / Backend (`fetchguard`) | — | AllowHosts for Discogs covers, PublicOnly for manual URLs. |
|
||||||
|
| Invitation mail | Background job (conga/River) | Mail (postcard) | Enqueued in the write transaction (D-13). |
|
||||||
|
| Realtime events (album broadcasts, user notifications) | Background job (lighthouse broadcast job) | Centrifugo | After-commit publication (P11 D-06). |
|
||||||
|
| Rate limits | API / Backend (surf inline `throttle:N,M`) | — | Per-route limits copied from routes.php. |
|
||||||
|
|
||||||
|
<user_constraints>
|
||||||
|
## User Constraints (from CONTEXT.md)
|
||||||
|
|
||||||
|
### Locked Decisions
|
||||||
|
|
||||||
|
#### Route boundary
|
||||||
|
- **D-01:** Collections, household and album routes listed in the domain are ported whole. Each handler is mounted once and reused on both groups wherever PHP mirrors it (the P6 D-15 genres pattern), with the exact `->where()` constraints and inline throttles (`throttle:10,1` on switch, share regenerate, invitation store and resend; `throttle:20,1` on album photo upload; `throttle:60,1` on albums/sync).
|
||||||
|
- **D-02:** `albums/sync`, `stats`, `value`, `missing` and `bulk` are ported here: same controllers, no external service, and the Nuxt Albums UI calls them. `bulk` is where the single `collection.bulk_updated` broadcast under `WithoutBroadcasting[Album]` (P11 D-08) is exercised on a real endpoint.
|
||||||
|
- **D-03:** Reservations move to Phase 13. In PHP they only exist as wishlist routes (`AlbumReservationService`, `WishlistAlbumReservationController`) and need the wishlist resolver. Fix the Phase 12 success criterion 2 and API-02 wording, and add them to Phase 13 / API-03.
|
||||||
|
- **D-04:** The Discogs cover-price route moves to Phase 14 with INTG-01. The Discogs routes that match or use AI stay in Phase 14. Their manifest entries stay `pending` (P6 D-15: no 501 shells). Fix success criterion 2 and API-02 wording.
|
||||||
|
- **D-05:** `CoverImporter` (the host-locked cover fetch behind `cover_urls` on album create and bulk) is ported now on the existing fetchguard `AllowHosts` GET helper (P6 D-11), so `POST albums` with `cover_urls` is parity-true in this phase. The pending todo `fetchguard-guarded-http-client.md` (POST, multipart, bearer) is **not** folded. It stays for Phase 14. If research finds that `CoverImporter` depends on `DiscogsClient` or needs more than a guarded GET, it reports that before planning.
|
||||||
|
- **D-06:** All anonymous public-share routes move to Phase 13. Phase 12 ports only the owner-only JWT `collection/share` surface: show, update and regenerate, which emits the `public_token`. API-01 and success criterion 1 ("public token views") are reworded at plan time, and `public/{token}*` added to Phase 13 alongside `public-wishlist/{token}*`.
|
||||||
|
- **D-07:** The manual cover URL branch of album photo upload ("exactly one of file or cover_url") uses fetchguard `PublicOnly`, as in PHP (P6 D-11/D-13).
|
||||||
|
|
||||||
|
#### Parity evidence
|
||||||
|
- **D-08:** New recordings against the isolated PHP instance, using the Phase 2 `tide` capture rules (private 0600 vars, no live tokens in git):
|
||||||
|
- `nuxt-collections` flow: create → switch → `me/context` → share show/update/regenerate → invite → accept as a second user → members → remove member.
|
||||||
|
- `nuxt-albums` flow: create (with and without `cover_urls`) → update → rate / unrate → photo upload and delete → search → stats/value/missing/sync → bulk → delete.
|
||||||
|
- Plus every distinct error status and body per route: 404 for foreign or missing ids, 403/404 non-owner, 410 on an unavailable invitation, 422 validation, and the personal-token 404 on owner-only household actions.
|
||||||
|
|
||||||
|
Envelopes are reproduced per endpoint (P7 D-13).
|
||||||
|
- **D-09:** Centrifugo publications are recorded from PHP during the album flows (create, update, delete, bulk) with the P11 D-10 capture tooling. The Go side, on the memory driver or a fake Centrifugo, is diffed against them with `timestamp`/`actor` normalised.
|
||||||
|
- **D-10:** Token-group twins run the same handler as the JWT route and each replays its own recorded fixture. A route-table test asserts that every twin carries the right `inv.scope:read|write` and that JWT-only routes (switch, share, household, invitations) are absent from the token group.
|
||||||
|
- **D-11:** Uploads get both recorded multipart replays (same file bytes, body diff including `thumb_url`) and Go tests. The Go tests assert the `system_files` row, the blob and the thumb exist; that the `ImageContentGuard` port rejects non-images; and that the upload-group `MaxBytesReader` cap holds (P7 D-04 pattern).
|
||||||
|
- **D-12:** Search replays run with search disabled, as PHP was recorded (`SCOUT_DRIVER=null`). Typesense behaviour is covered by Go tests (D-16..D-18).
|
||||||
|
|
||||||
|
#### Invitations and mail
|
||||||
|
- **D-13:** Invitation mail is sent by a River job enqueued **in the same transaction** as the invitation write, using the P11 job machinery. It goes out only if the invitation commits, which matches PHP's commit-before-side-effect ordering without the inline send.
|
||||||
|
- The raw token (64 hex chars, sha256 at rest, 7-day expiry, as PHP) is present only in the job args and the mail. It is never logged and never stored in `summer_jobs.metadata`.
|
||||||
|
- Locale and link follow PHP: inviter `preferred_locale` picks `collection_invitation` / `collection_invitation-en` and `/zaproszenia/{token}` / `/en/invitations/{token}` under `app.url`.
|
||||||
|
- Asserted through postcard's `memory` driver.
|
||||||
|
- **D-14:** Accepting an invitation ports `NotificationService`'s write path now: the `notifyInvitationAccepted` row, plus any mail it sends, through the same job mechanism. The recorded accept flow then matches PHP DB state. Phase 13 adds the notifications list, read and prune API on top.
|
||||||
|
- **D-15:** Only the existing-account path is ported. That means invite, resend and cancel for any email; accept by a logged-in user; and removing an editor with PHP's `CollectionProvisioner` context repair and organisation-membership rules. PendingInvitationRegistration rows are cleared on accept as PHP does, but consuming them on register belongs to Phase 13.
|
||||||
|
|
||||||
|
#### Search and the security test
|
||||||
|
- **D-16:** Scout semantics are mirrored exactly. When Typesense is on and the query takes the Typesense path:
|
||||||
|
1. Typesense returns one page of candidate ids.
|
||||||
|
2. SQL applies `whereIn(ids)` + `accessibleBy(user)` + active collection (or token-frozen collection) + filters.
|
||||||
|
3. `meta.total`/`last_page` come from Typesense, as Scout's paginator does. A stale document therefore yields a short page, never a leaked row. The inflated total is a documented, tested quirk, not a bug to fix.
|
||||||
|
|
||||||
|
Rating, name and artist sorts force the SQL path, as PHP does. A Typesense error logs a warning and falls back to the SQL search.
|
||||||
|
- **D-17:** The SQL text search uses `ILIKE` with `%` and `_` escaped in user input, over PHP's column lists (`TEXT_FIELDS_AUTHENTICATED`: name, notes, artist_display, track_titles, label, catalog_number, plus artists.name). Escaping is a small, deliberate hardening: PHP passes `%`/`_` through, so results differ only for queries containing them. Polish ordering relies on the `pl-PL` ICU database (P3 D-06).
|
||||||
|
- **D-18:** The leak security test (success criterion 3) proves, with a scripted fake engine driver returning poisoned ids through the P11 search interface, that none of these return a row:
|
||||||
|
1. A stale document whose album moved to a collection the caller cannot access.
|
||||||
|
2. A mis-scoped document carrying the caller's `collection_id` for a foreign album.
|
||||||
|
3. A soft-deleted album still in the index.
|
||||||
|
4. An editor removed from a collection whose documents remain.
|
||||||
|
5. A personal token frozen to its creation-time collection receiving hits from the user's other collections.
|
||||||
|
|
||||||
|
No Typesense container is used.
|
||||||
|
|
||||||
|
#### Carried forward (locked earlier)
|
||||||
|
- **C-01:** `ActiveCollection` (P3 D-03) is extended with switching and editor membership. `me/context` never exposes a raw `collection_id`, and the channel name is opaque (P11 D-11 naming rules).
|
||||||
|
- **C-02:** Write paths go through the ported `AlbumWriteService` / `CollectionWriteService` fill boundaries (P5 D-05..D-07). The new request-DTO fuzz covers every write endpoint of this phase (unknown and server-owned keys never persisted). This inherits P5 criterion 3's HTTP half.
|
||||||
|
- **C-03:** Response DTOs come from ported `Serialize*` functions, never from marshalled models. Use `[]` not `null`, Carbon `+00:00` times, and the pagination envelope without `links` (P5 D-08, P6 D-17, DATA-10).
|
||||||
|
- **C-04:** Album broadcasts and Typesense sync are already wired by Phase 11 (after-commit inline sync, broadcast job in the write transaction). This phase only calls them.
|
||||||
|
|
||||||
|
### Claude's Discretion
|
||||||
|
- Handler file layout under `controllers/api` and how the ~800-line `AlbumApiController` is split across Go files.
|
||||||
|
- DTO structs and how the fuzz enumerates write endpoints (route-table driven preferred).
|
||||||
|
- Job kind names and queue for invitation and notification mail.
|
||||||
|
- Fake engine driver shape for tests.
|
||||||
|
- Which errors per route need a recording and which a Go test with bodies from PHP source, where recording a case is impractical (e.g. a mid-transaction race). Recording is the default.
|
||||||
|
- Plan count and split, subject to the plan-count checkpoint, "unit tests are the last plan" and the security-review agent (this phase touches authorization and public tokens).
|
||||||
|
|
||||||
|
### Deferred Ideas (OUT OF SCOPE)
|
||||||
|
- Reservations (`wishlist/albums/{id}/reserve|reveal`) → Phase 13.
|
||||||
|
- Anonymous public-share routes `public/{token}*` → Phase 13.
|
||||||
|
- Discogs cover-price, release match, apply-release, recognize, Discogs import → Phase 14.
|
||||||
|
- Register-via-invitation and public invitation inspection → Phase 13 (API-07).
|
||||||
|
- Notifications list, read and prune API → Phase 13 (API-04).
|
||||||
|
|
||||||
|
#### Reviewed Todos (not folded)
|
||||||
|
- `fetchguard-guarded-http-client.md`: not folded. Discogs cover-price moved to Phase 14, and `CoverImporter` only needs the existing guarded GET.
|
||||||
|
- `backend-admin-api-tokens.md`: unrelated to this API surface (admin auth). Stays deferred past v1.
|
||||||
|
- `redacting-slog-handler.md`: Phase 14 (credentials). Worth noting that invitation tokens must not reach logs (D-13), but no handler port is needed here.
|
||||||
|
</user_constraints>
|
||||||
|
|
||||||
|
<phase_requirements>
|
||||||
|
## Phase Requirements
|
||||||
|
|
||||||
|
| ID | Description | Research Support |
|
||||||
|
|----|-------------|------------------|
|
||||||
|
| API-01 | Collections: CRUD, active-context switch (me/context returns an opaque channel name), editor invitations and acceptance, share link regeneration, public token views | The collections, switch, share, household and invitations sections below. Correction: the channel name comes from `GET realtime/channels`, not `me/context` (see Finding 2). "Public token views" moves to Phase 13 (D-06). |
|
||||||
|
| API-02 | Albums: CRUD, ratings, reservations, photo upload and manual cover URL, Discogs cover price, artists/genres/styles lookups, search that treats Typesense as a pre-filter re-gated in SQL | The albums, uploads, search and lookups sections. Reservations move to Phase 13 (D-03) and Discogs cover price to Phase 14 (D-04). The search total semantics are corrected in Finding 1. |
|
||||||
|
</phase_requirements>
|
||||||
|
|
||||||
|
## Project Constraints (from CLAUDE.md)
|
||||||
|
|
||||||
|
- Lean planning: few, large plans. Present the plan count with one-line scopes before writing any PLAN.md, then wait for confirmation.
|
||||||
|
- The last plan of every phase brings full unit-test coverage. Earlier plans may carry smoke tests.
|
||||||
|
- Use the standard library first. Add a dependency only when the research doc or a phase decision names it. `go vet` and `go test ./...` stay green at every commit.
|
||||||
|
- Plugins are compiled in. No runtime plugin loading.
|
||||||
|
- **API parity is the acceptance test.** Don't "improve" response shapes.
|
||||||
|
- Two repos: `fonoteka.go` holds the app and `summercms.go` the framework. The framework never names the app in READMEs or docs. Planning docs stay in `summercms.go/.planning`.
|
||||||
|
- Any change to a module's exported API, config keys, CLI commands or dependencies updates that module's `README.md` and the affected `docs/` pages in the same change (`go test ./cmd/summer -run TestDocsTree`, `summer docs:build --check`). Every identifier named in docs must exist.
|
||||||
|
- Commits: one logical change each, no co-author tags, planning docs and code in separate commits.
|
||||||
|
- Core plugin contracts (user, blog, pages, payment) stay unbroken. This bears on any change to the Go `golem15.user` plugin, for example user groups (see Open Question 3).
|
||||||
|
- The GORM, River, fetchguard, lighthouse, beachcomber, postcard and conga choices are already decided. Don't swap them.
|
||||||
|
|
||||||
|
## Critical Findings (read before planning)
|
||||||
|
|
||||||
|
### Finding 1: The search total is re-gated in SQL (D-16 point 3 does not hold)
|
||||||
|
Read this session from `vendor/laravel/scout/src/Builder.php` lines 533-556 [VERIFIED: /media/nvme/dev/golem15/fonoteka/vendor/laravel/scout/src/Builder.php:533-556]:
|
||||||
|
```php
|
||||||
|
$totalCount = $engine->getTotalCount($results);
|
||||||
|
if (is_null($this->queryCallback)) {
|
||||||
|
return $totalCount;
|
||||||
|
}
|
||||||
|
$ids = $engine->mapIdsFrom($results, $this->model->getScoutKeyName())->all();
|
||||||
|
if (count($ids) < $totalCount) {
|
||||||
|
$ids = $engine->keys(tap(clone $this, function ($builder) use ($totalCount) {
|
||||||
|
$builder->take(
|
||||||
|
is_null($this->limit) ? $totalCount : min($this->limit, $totalCount)
|
||||||
|
);
|
||||||
|
}))->all();
|
||||||
|
}
|
||||||
|
return $this->model->queryScoutModelsByIds(
|
||||||
|
$this, $ids
|
||||||
|
)->toBase()->getCountForPagination();
|
||||||
|
```
|
||||||
|
The installed version is `"version": "v10.25.0"` [VERIFIED: vendor/composer/installed.json]. `AlbumSearchService::searchWithScout` always sets `->query(...)`, so `queryCallback` is never null. The Typesense engine has `private int $maxPerPage = 250;` [VERIFIED: Engines/TypesenseEngine.php:41]. A limit of 250 or more switches to `performPaginatedSearch`, which loops up to `$this->maxTotalResults`. That value defaults to `$config['max_total_results'] ?? 1000` [VERIFIED: EngineManager.php:165].
|
||||||
|
|
||||||
|
What PHP actually returns on the Typesense path:
|
||||||
|
- **Items:** the Typesense page's ids, re-gated in SQL (`queryScoutModelsByIds`: the callback runs, then `whereIntegerInRaw(id, ids)`), filtered and ordered by Typesense position. Stale or foreign ids drop out, so the page can be short.
|
||||||
|
- **`meta.total`:** when `count(page ids) < found`, a second Typesense search fetches `min(found, 1000)` ids (in pages of 250 once found is 250 or more). The total is then `COUNT(*)` of those ids with the same callback applied. When found fits in one page, it is the re-gated count of that page.
|
||||||
|
- **`last_page`** is `ceil(total/per_page)`.
|
||||||
|
- **Quirk worth mirroring and testing:** the total is capped at 1000 even when more albums match.
|
||||||
|
- **Soft deletes:** `queryScoutModelsByIds` calls `withTrashed()` only when the model uses Illuminate's `SoftDeletes` [VERIFIED: Searchable.php:322-339, 507-510]. Album uses `\Winter\Storm\Database\Traits\SoftDelete` [VERIFIED: models/Album.php:18], so Winter's soft-delete scope still applies and soft-deleted albums never come back.
|
||||||
|
|
||||||
|
**Action:** tell the user that D-16 step 3 should read "total = re-gated SQL count over min(found, 1000) Typesense ids", which is what "mirror Scout exactly" means here. The D-18 leak test then also asserts that `meta.total` excludes poisoned ids. The Go engine has to expose `found` (see Finding 5).
|
||||||
|
|
||||||
|
### Finding 2: `me/context` carries no channel name; `realtime/channels` does
|
||||||
|
The recorded PHP `me/context` body [VERIFIED: parity/fixtures/routes/GET___fonoteka_api_v1_me_context_jwt.yaml]:
|
||||||
|
`{"can_use_ai":false,"can_manage_org":true,"has_organisation":true,"ai_org_lock":false,"ai_inherited":false,"is_collection_owner":true,"invitation_acceptance_required":false,"can_import_discogs":false,"market_currency_default":"EUR","market_currencies":["USD","GBP","EUR","CAD","AUD","JPY","CHF","MXN","BRL","NZD","SEK","ZAR"]}`
|
||||||
|
|
||||||
|
The channel route [VERIFIED: routes.php:94-99] is `Route::get('realtime/channels', ... 'collection' => 'collection:' . app(ActiveCollectionResolver::class)->resolve(...)->id`. It is recorded as `{"data":{"collection":"collection:{{id:style}}"}}` and is `status: pending` in the manifest [VERIFIED: parity/manifest.yaml:93-97]. The Nuxt app calls `/realtime/channels` [VERIFIED: grep of vue-fonoteka-app/app].
|
||||||
|
|
||||||
|
`me/context` needs ports of `AiGate::allows`, `OrgAccess::canManage`, `DiscogsGate::allows` (→ `DiscogsConfigResolver`: site admin → env token, then user credential, then org credential), `config('fonoteka.ai_org_lock')`, and the `OrgAiCredential`/`UserAiCredential` existence checks [VERIFIED: MeContextController.php, AiGate.php, DiscogsConfigResolver.php].
|
||||||
|
|
||||||
|
**Action:** add `GET realtime/channels` (JWT only) to this phase as the "opaque channel name" half of API-01, and port the gates as read-only predicates. The channel string embeds the raw collection id, exactly as PHP does. Parity wins over P11 D-11's "no raw ids", and the PHP comment explains why that is safe: the authorizer re-validates membership on every subscribe.
|
||||||
|
|
||||||
|
### Finding 3: HttpException error bodies are Winter HTML pages
|
||||||
|
`ActiveCollectionResolver::switchTo`, `InvitationApiController::ownerCollection`, `CollectionMemberController`, `InvitationService::notFound()` and `accept()` throw `HttpException`, which Winter renders as HTML. Recorded examples:
|
||||||
|
- The members 404 is the Polish `Nie znaleziono strony` page with `Content-Type: "text/html; charset=UTF-8"`.
|
||||||
|
- The accept 410 was recorded as the **debug** exception page, with PHP file paths and `line 140` [VERIFIED: fixtures/routes/DELETE___fonoteka_api_v1_household_members_{id}_jwt.yaml, POST___fonoteka_api_v1_invitations_{token}_accept_jwt.yaml].
|
||||||
|
|
||||||
|
`php_parity.sh` now exports `export APP_DEBUG=false` [VERIFIED: parity/php_parity.sh:49]. **Re-record** every HttpException case. The P7/P8 precedent embeds the production page bytes (`controllers/api/winter_error_page.html`, which contains the literal `http://127.0.0.1:8423/modules/system/assets/css/styles.css`) [VERIFIED: fonoteka.go/plugins/golem15/fonoteka/controllers/api/winter_error_page.html].
|
||||||
|
|
||||||
|
Two more envelopes still need a recording, because they are not the controller's own JSON shape:
|
||||||
|
- `household/invitations` store validates with `$request->validate([...])`, which throws `ValidationException`.
|
||||||
|
- `normalizeEmail` throws `ValidationException::withMessages(['email' => 'The email must be a valid email address.'])`.
|
||||||
|
|
||||||
|
### Finding 4: The photo URL prefix differs from PHP
|
||||||
|
- **PHP:** `getPublicPath()` returns `Config::get('cms.storage.uploads.path', '/storage/app/uploads')` plus `'/public'` for public files [VERIFIED: modules/system/models/File.php:63-75], with `'path' => '/storage/app/uploads'` [VERIFIED: config/cms.php:322].
|
||||||
|
- **Go:** `const defaultPublicPathPrefix = "/storage/uploads"` [VERIFIED: modules/lagoon/attach/bucket.go], the app config has `public_path_prefix: "/storage/uploads"` and `bucket_url: "file://./storage/app/uploads"` [VERIFIED: fonoteka.go/config/storage.yaml], and `BlobKey` = partition + disk name with no `public/` segment [VERIFIED: attach/thumb.go].
|
||||||
|
|
||||||
|
P5 D-16 required the exact PHP partition path `public/xxx/yyy/zzz/<disk_name>` and URL shape. **Action:** set `bucket_url: file://./storage/app/uploads/public` and `public_path_prefix: /storage/app/uploads/public` in the app config (or add a `public/` sub-prefix in attach). Verify that the user avatar URLs (the same bucket) stay correct, and pin the shape with a test. There is also no exported URL helper for an original file (`publicURL` is unexported), so `serializePhoto.url` needs a small `attach` export, for example `(*File).URL()`.
|
||||||
|
|
||||||
|
### Finding 5: Framework gaps that block parity (summercms.go)
|
||||||
|
|
||||||
|
| Gap | Where | Needed for |
|
||||||
|
|-----|-------|-----------|
|
||||||
|
| `Engine.SearchIDs` returns ids only, so there is no `found`. `Query` has no `QueryByWeights` | `beachcomber/searchable.go` (`SearchIDs(ctx, index string, q Query) ([]string, error)`) [VERIFIED] | Finding 1's total, and the weights PHP sends (`'query_by_weights' => '10,10,5,5,3,1,3,3'`) |
|
||||||
|
| Requests can't carry multipart bodies (`Body string` only). Responses have `BodyFile`, requests don't | `tide/flow.go` `Request{Method, Path, Query, Headers, Body}` [VERIFIED] | D-11 recorded multipart replays |
|
||||||
|
| No normalizer for random `disk_name` in `url`/`thumb_url` | `tide/normalize.go` masks only id/date/`collection_key`/`client_id` keys [VERIFIED] | Upload replays and album bodies that carry photos |
|
||||||
|
| The album subtree's `created_at`/`updated_at` in publication goldens may not be masked (README says only `data.timestamp`, actor and ids) | `tide` `NormalizePublications` [CITED: parity/README.md "Broadcast goldens"] | Flipping the `created`/`updated` goldens from pending to assertion |
|
||||||
|
| No validation for `array`, `*` wildcards, `url`, `date`, `after_or_equal`, `before_or_equal`, `exists`, `regex`, `string`, `image`, `sometimes`, closure rules. The size-type messages (`max.string`/`max.array`/`max.numeric`/`max.file`) aren't split. The implicit-rule stop isn't implemented. The pl/en catalogs hold 13 keys | `lagoon/validate.go`, `phrasebook/lang/{pl,en}/validate.yaml` [VERIFIED] | Every 422 body. Winter's source catalog is `modules/system/lang/pl/validation.php` [VERIFIED] |
|
||||||
|
| The `min` failure message bug (pending todo `lagoon-validate-min-message.md`) | `lagoon/validate.go` | `market_price_stored min:0`, `year between`, `rating min:1|max:5`. Fold this todo. |
|
||||||
|
| No exported original-file URL | `lagoon/attach` | `serializePhoto.url` |
|
||||||
|
| fetchguard's test seams (`tlsConfig`, `skipReservedCheck`) are unexported | `fetchguard/policy.go` [VERIFIED] | App tests for the cover fetch success path. Use an injectable fetch func in the app instead of changing fetchguard. |
|
||||||
|
|
||||||
|
## Standard Stack
|
||||||
|
|
||||||
|
### Core (all already in the tree; no new framework choices)
|
||||||
|
| Library / module | Version | Purpose | Why Standard |
|
||||||
|
|---------|---------|---------|--------------|
|
||||||
|
| `net/http` ServeMux via `surf` | Go 1.27 | Routing, `Where` constraints, inline `throttle:N,M`, `body.limit` | Project rule; P6 [VERIFIED: surf/router.go] |
|
||||||
|
| GORM + `lagoon` | gorm v1.31.2 | Scopes, transactions (`lagoon.Transaction`), `Fill`, `Paginate`, `OrderBy`+`Collate` | Decided [CITED: CLAUDE.md stack] |
|
||||||
|
| `lagoon/attach` | framework | `system_files` rows, blob keys, `(*File).Thumb(ctx, bucket, w, h, mode)` with `crop` | P5 D-14..D-17 [VERIFIED: attach/thumb.go] |
|
||||||
|
| `fetchguard` | framework | `Fetch(ctx, url, Policy{Mode: AllowHostsMode/PublicOnlyMode, AllowHosts, MaxBytes, Timeout}, cfg)` with the `invalid_url/scheme/unresolvable/private_ip/network_error/too_large` reasons | P6 D-11 [VERIFIED: fetchguard/fetch.go, policy.go] |
|
||||||
|
| `beachcomber` | framework | `Engine.SearchIDs` and the settings gate | P11 D-19 [VERIFIED] |
|
||||||
|
| `lighthouse` | framework | `WithoutBroadcasting[T](ctx, fn)`, `(*Service).Emit(ctx, db, Broadcast{Channels, Event, Payload})` | P11 D-06/D-08 [VERIFIED: lighthouse] |
|
||||||
|
| `conga` | framework | `conga.Job[T](fn, OnQueue/MaxAttempts/Timeout)`, `(*Manager).Enqueue(ctx, tx, args, EnqueueOpts)` (joins the tx) | P11 D-02 [VERIFIED: conga/conga.go] |
|
||||||
|
| `postcard` | framework | `Mailer.Send(ctx, postcard.Message{Template, To, Vars})`, `MemoryDriver` for tests | P7 precedent [VERIFIED: user/controllers/api_controller.go:902] |
|
||||||
|
| `phrasebook` | framework | Validation catalogs (pl/en) | P5 [VERIFIED] |
|
||||||
|
| `tide` | framework | Recording, replay and broadcast goldens | P2/P11 |
|
||||||
|
| `crypto/rand`, `crypto/sha256`, `crypto/hmac`, `crypto/subtle` | stdlib | Share token, invite token, fingerprint, constant-time compare | stdlib |
|
||||||
|
|
||||||
|
### Supporting
|
||||||
|
| Library | Version | Purpose | When to Use |
|
||||||
|
|---------|---------|---------|-------------|
|
||||||
|
| `golang.org/x/image/webp` | bump `golang.org/x/image` from the transitive `v0.0.0-20191009234506-e7c1f5e7dbb8` to `v0.46.0` [VERIFIED: `go list -m golang.org/x/image@latest` → v0.46.0, 2026-09-08] | Register the webp decoder so `image.DecodeConfig` (the `getimagesize` stand-in) and thumbs work for `.webp` | Only if webp uploads must decode. PHP accepts `mimes:jpg,jpeg,png,gif,webp`. Name it in a phase decision. It is official Go, already in the module graph. |
|
||||||
|
|
||||||
|
### Alternatives Considered
|
||||||
|
| Instead of | Could Use | Tradeoff |
|
||||||
|
|------------|-----------|----------|
|
||||||
|
| Extending `beachcomber.Engine` with a `found` count | An optional interface (e.g. `PagedSearcher`, `SearchPage(ctx, index, q) (ids []string, found int, err error)`) type-asserted by the app | No breaking change for other engine implementers (null, typesense, test fakes). **Recommended.** |
|
||||||
|
| A full Laravel validator port | Hand-written per-endpoint checks | Hand-written checks drift on message text and ordering. One validator with Laravel semantics is reused through Phase 13. |
|
||||||
|
| River job for invite mail | Inline send after commit | Locked by D-13 (job) |
|
||||||
|
|
||||||
|
**Installation:** none for the core. The optional webp support is `go get golang.org/x/image@v0.46.0` in `summercms.go` (where attach lives) after a phase decision names it.
|
||||||
|
|
||||||
|
## Package Legitimacy Audit
|
||||||
|
|
||||||
|
| Package | Registry | Age | Downloads | Source Repo | Verdict | Disposition |
|
||||||
|
|---------|----------|-----|-----------|-------------|---------|-------------|
|
||||||
|
| golang.org/x/image | Go module proxy | ~10 yrs | n/a (Go sub-repository) | go.googlesource.com/image | OK (official Go project, already a transitive dependency) | Approved, conditional on the webp decision |
|
||||||
|
|
||||||
|
The `gsd-tools package-legitimacy` seam covers npm/pypi/crates only. The Go module was checked with `go list -m` against proxy.golang.org.
|
||||||
|
**Packages removed due to [SLOP] verdict:** none.
|
||||||
|
**Packages flagged as suspicious [SUS]:** none.
|
||||||
|
|
||||||
|
## Architecture Patterns
|
||||||
|
|
||||||
|
### System Architecture Diagram
|
||||||
|
|
||||||
|
```
|
||||||
|
HTTP request
|
||||||
|
│
|
||||||
|
├─ /_fonoteka/api/v1/* ──► jwt.auth ─► locale.from-principal ─► inv.must-change-password ─► [route throttle] ─┐
|
||||||
|
└─ /api/v1/fonoteka/* ──► inv_token ─► throttle:fonoteka-api-token ─► inv.scope:<read|write> (per route) ──┤
|
||||||
|
▼
|
||||||
|
handler (one per route, shared by twins)
|
||||||
|
│
|
||||||
|
┌──────────── request decode (JSON / multipart / query) ◄─ body.limit
|
||||||
|
▼
|
||||||
|
Laravel-semantics validator ──fail──► 422 {"error":"Validation failed","errors":{...}} (pl/en)
|
||||||
|
│ ok
|
||||||
|
▼
|
||||||
|
tenant resolution: token pin? ─yes─► pinned collection (404 if not accessible)
|
||||||
|
│ no
|
||||||
|
▼
|
||||||
|
pending-invitation guard (409) ─► users FOR UPDATE ─► context FOR UPDATE ─► stored|fallback|provision
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
AccessibleBy(user, token) scope + collection_id re-gate ──none──► 404 JSON / Winter HTML page
|
||||||
|
│
|
||||||
|
┌────────────────────────┼─────────────────────────────┬───────────────────────────┐
|
||||||
|
▼ ▼ ▼ ▼
|
||||||
|
write service (Fill search: gate on? sort forces uploads: content guard, invitations: token gen,
|
||||||
|
allow-list, tx, SQL? ─► engine SearchPage ─► blob write, system_files sha256 at rest, tx ─►
|
||||||
|
WithoutBroadcasting) ids ─► SQL whereIn + scopes row, Thumb(200,200,crop) conga.Enqueue(mail job)
|
||||||
|
│ ─► recount (Scout) manual URL: fetchguard ─► commit ─► River ─►
|
||||||
|
│ │ error ─► SQL ILIKE path PublicOnly; cover_urls: postcard
|
||||||
|
▼ ▼ AllowHosts discogs.com
|
||||||
|
commit ─► after-commit: beachcomber sync, lighthouse broadcast job (Emit single event) ─► Centrifugo
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Serialize* DTO (never the model) ─► JSON (Cache-Control: no-cache, private)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Recommended Project Structure (fonoteka.go)
|
||||||
|
```
|
||||||
|
plugins/golem15/fonoteka/
|
||||||
|
├── classes/
|
||||||
|
│ ├── active_collection.go # Resolve (token pin, guard, provision), SwitchTo, DisplayActiveID
|
||||||
|
│ ├── access.go # AccessibleBy(user, token) = membership AND token narrowing
|
||||||
|
│ ├── collection_provisioner.go # CollectionProvisioner, OrgProvisioner
|
||||||
|
│ ├── fingerprint.go # CollectionFingerprint (HMAC app.key)
|
||||||
|
│ ├── share_service.go # CollectionShareService (owner surface only)
|
||||||
|
│ ├── invitation_service.go # send/resend/cancel/accept/removeEditor
|
||||||
|
│ ├── notification_service.go # write(), notifyInvitationAccepted, notifyAlbumAdded
|
||||||
|
│ ├── gates.go # OrgAccess, AiGate, DiscogsGate (read-only)
|
||||||
|
│ ├── serialize.go # full SerializesFonoteka port (replace stubs)
|
||||||
|
│ ├── album_write_service.go # create/apply (+styles, artists, tracklist_text, created_at, covers, completeness)
|
||||||
|
│ ├── artist_resolver.go # full ArtistResolver + displayFor
|
||||||
|
│ ├── tracklist_text_parser.go, added_date_parser.go, duplicate_matcher.go, completeness.go
|
||||||
|
│ ├── album_search.go # AlbumSearchService (Typesense path + SQL path)
|
||||||
|
│ ├── album_sync.go # AlbumSyncService (delta, tombstones, cursor)
|
||||||
|
│ ├── image_guard.go, cover_importer.go, manual_cover_fetcher.go
|
||||||
|
├── controllers/api/
|
||||||
|
│ ├── collections.go, collection_share.go, me_context.go, realtime_channels.go
|
||||||
|
│ ├── invitations.go, members.go
|
||||||
|
│ ├── albums.go, albums_bulk.go, albums_photos.go, albums_stats.go (stats/value/missing), albums_sync.go, ratings.go
|
||||||
|
│ ├── lookups.go (artists, styles, genres POST)
|
||||||
|
│ └── errors.go (Winter HTML pages, notFound helpers)
|
||||||
|
├── jobs.go # pact.HasJobs: invitation mail job, notification mail job
|
||||||
|
├── views/mail/collection_invitation{,-en}.htm (postcard templates)
|
||||||
|
└── routes.go # line-by-line from routes.php
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pattern 1: Token-aware tenant resolution (ports ActiveCollectionResolver)
|
||||||
|
**What:** `Resolve(ctx, tx, user, token *models.ApiToken)`:
|
||||||
|
- **Token present:** require `len(token.CollectionIDs)==1 && id>=1`, then load it through `AccessibleBy`. Anything else is HttpException 404 `Collection not found`, a Winter HTML page.
|
||||||
|
- **No token:** run `guardPendingInvitation` (409 `invitation_acceptance_required`), then in one transaction `SELECT users ... FOR UPDATE`, then the context `FOR UPDATE`, then stored-if-accessible, else the lowest-id accessible collection, else provision `Moja kolekcja`.
|
||||||
|
|
||||||
|
`SwitchTo` returns 404 for a token caller and adds `kind='collection'` [VERIFIED: ActiveCollectionResolver.php:20-123].
|
||||||
|
**Note:** PHP's fallback in `resolve()` uses `Collection::accessibleBy($user)->orderBy('id')->first()` with **no kind filter** (line 46), while the current Go `firstAccessibleRealCollection` adds `kind = collection`. Check which one parity needs (a wishlist with a lower id than the real collection would differ). This is an [ASSUMED] risk; record a case.
|
||||||
|
**Token source:** `cred, _ := bouncer.Credential(r.Context()); token, _ := cred.(*models.ApiToken)` [VERIFIED: middleware/token_scope.go].
|
||||||
|
|
||||||
|
### Pattern 2: One access chokepoint with token narrowing
|
||||||
|
```go
|
||||||
|
// Ports Collection::scopeAccessibleBy (models/Collection.php:164-200).
|
||||||
|
func AccessibleBy(userID uint, token *models.ApiToken) func(*gorm.DB) *gorm.DB {
|
||||||
|
return func(db *gorm.DB) *gorm.DB {
|
||||||
|
db = AccessibleByMembership(userID)(db) // grouped owner OR editor
|
||||||
|
if token != nil && len(token.CollectionIDs.Get()) > 0 {
|
||||||
|
db = db.Where(`(golem15_fonoteka_collections.id IN ? OR (golem15_fonoteka_collections.owner_id = ? AND golem15_fonoteka_collections.kind = 'wishlist'))`,
|
||||||
|
token.CollectionIDs.Get(), userID)
|
||||||
|
}
|
||||||
|
return db
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Album::scopeAccessibleBy = whereHas('collection', accessibleBy): join or EXISTS on collections with deleted_at IS NULL.
|
||||||
|
```
|
||||||
|
`ApiToken.CollectionIDs lagoon.Jsonable[[]uint]` [VERIFIED: models/api_token.go:17].
|
||||||
|
|
||||||
|
### Pattern 3: Scout-exact search (Finding 1)
|
||||||
|
```go
|
||||||
|
// Typesense path (gate on, sort not in {rating,name,artist,price}, no rating filter):
|
||||||
|
ids, found, err := engine.SearchPage(ctx, index, beachcomber.Query{Q: q, QueryBy: ..., QueryByWeights: ..., FilterBy: "collection_id:=[<id>] && ...", SortBy: sortBy, Page: page, PerPage: perPage})
|
||||||
|
if err != nil { log.Warn(...); return sqlPath() }
|
||||||
|
items := hydrate(ids) // SQL: scope(user,token) + collection_id + embeds + id IN ids; keep Typesense order
|
||||||
|
total := len(reGate(ids))
|
||||||
|
if len(ids) < found {
|
||||||
|
all := fetchKeys(min(found, 1000)) // per_page=found if <250, else pages of 250 up to 1000
|
||||||
|
total = countReGated(all)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
The filter strings are verbatim from PHP: `'collection_id:=[' . $collectionId . ']'`, `'genre_id:='`, `'style_ids:='`, `'format:='`, `'medium:='`, `'artist_ids:='`, `'year:[' . $decade . '..' . ($decade + 9) . ']'`, joined by `' && '`. The sort is `($sort === 'year' ? 'year' : 'created_at') . ':' . $dir`, prefixed with `'_text_match:desc,'` when q is non-empty [VERIFIED: AlbumSearchService.php:174-199].
|
||||||
|
|
||||||
|
### Pattern 4: Single album broadcast (store/update) and bulk
|
||||||
|
- **Store/update:** PHP wraps the whole write in `Album::withoutBroadcasting`, then publishes exactly one `created.fonoteka.album` / `updated.fonoteka.album` with `getBroadcastPayload` (fresh load with `genre, styles, artists, photos`) [VERIFIED: AlbumApiController.php:89-98, 489-506; models/Album.php:450-467]. In Go, use `lighthouse.WithoutBroadcasting[models.Album]` and one `svc.Emit`, with a payload built by the same `albumPayload` shape that `realtime.go` already defines, now on the full serializer.
|
||||||
|
- **Bulk:** emit `collection.bulk_updated` `{"reason":"bulk_create","count":N}` only when `created !== [] && $active->kind === 'collection'` [VERIFIED: AlbumApiController.php:150-156].
|
||||||
|
|
||||||
|
### Pattern 5: Invitation mail as a transactional job (D-13)
|
||||||
|
In one `lagoon.Transaction`:
|
||||||
|
1. Generate the token with `hex(rand 32)`.
|
||||||
|
2. Upsert the invitation (`token_hash = sha256 hex`, `expires_at = now+7d`, `accepted_at/by = nil`, `revoked_at = nil`).
|
||||||
|
3. Call `conga.Manager.Enqueue(ctx, tx, InvitationMailArgs{InvitationID, Token}, …)`.
|
||||||
|
|
||||||
|
`OrgProvisioner.provisionFor(actor)` runs **inside** the same transaction [VERIFIED: InvitationService.php:48-72]. `Enqueue` (not `Dispatch`) avoids a `summer_jobs` row, so nothing can reach `summer_jobs.metadata`.
|
||||||
|
|
||||||
|
### Anti-Patterns to Avoid
|
||||||
|
- **Serializing models:** C-03, Pitfall 3. All responses go through `Serialize*`.
|
||||||
|
- **Treating a Typesense hit as authorization:** Pitfall 15. Always re-gate.
|
||||||
|
- **A bulk `tx.Where(...).Delete(&Album{})` for the collection cascade:** this is what the current `models.Collection.BeforeDelete` does [VERIFIED: models/collection.go:47-51]. `beachcomber.afterWrite` skips zero-PK reflect values [VERIFIED: beachcomber/sync.go:51-101], so no search document is removed and no per-album `deleted` broadcast fires. PHP deletes album by album (`foreach ($this->albums as $album) { $album->delete(); }`, [VERIFIED: models/Collection.php:127-134]). Load the albums and delete the slice instead.
|
||||||
|
- **Holding a transaction open across cover downloads:** up to `max_covers` (5) × a 10 s timeout. Commit the album first, then import covers and save `cover_import_failures`, then serialize and emit.
|
||||||
|
- **Trimming or nulling input strings:** the Winter kernel has neither `TrimStrings` nor `ConvertEmptyStringsToNull` [VERIFIED: vendor/winter/storm/src/Foundation/Http/Kernel.php:25-41]. The `CollectionShareController` comment claiming "the HTTP kernel rewrites an empty string to null" does not hold for this stack. Persist strings as sent.
|
||||||
|
|
||||||
|
## Don't Hand-Roll
|
||||||
|
|
||||||
|
| Problem | Don't Build | Use Instead | Why |
|
||||||
|
|---------|-------------|-------------|-----|
|
||||||
|
| SSRF-safe fetch | A custom `http.Client` with an IP check | `fetchguard.Fetch` (dial-time reserved-IP check, no redirects, byte cap) | DNS-rebinding and redirect edge cases are already solved and tested |
|
||||||
|
| Thumbnails | Your own resize or name scheme | `attach.(*File).Thumb(ctx, bucket, 200, 200, "crop")` | Winter filename parity (`thumb_<id>_<w>_<h>_0_0_<mode>.<ext>`) |
|
||||||
|
| Pagination envelope | A map literal | `lagoon.Paginate` / `lagoon.PageMeta` | DATA-10 shape, no `links` |
|
||||||
|
| Broadcast after commit | A direct publish | `lighthouse.Emit` inside the tx, `WithoutBroadcasting` | Rollback-safe (P11 D-06) |
|
||||||
|
| Job enqueue in a tx | Your own outbox | `conga.Manager.Enqueue(ctx, tx, ...)` | `InsertTx` on the same `*sql.Tx` [VERIFIED] |
|
||||||
|
| Random tokens | `math/rand`, uuid | `crypto/rand` + rejection sampling (`REJECT_AT = 248`) | Bias-free 62-symbol alphabet [VERIFIED: CollectionShareService.php:35-63] |
|
||||||
|
| Polish ordering | Your own sort | `lagoon.OrderBy(..., lagoon.Collate("pl-x-icu"))` | P3 D-06 |
|
||||||
|
| Validation messages | Inline strings | The phrasebook catalog ported from Winter `validation.php` (pl + en) | Locale-dependent recorded bodies |
|
||||||
|
|
||||||
|
**Key insight:** every "small" helper in this surface has a recorded byte contract (messages, URL shapes, ordering, key presence). Reuse the framework primitives and grow them where needed. Hand-rolled one-offs drift.
|
||||||
|
|
||||||
|
## Runtime State Inventory
|
||||||
|
|
||||||
|
Not a rename or migration phase. Runtime state still bears on parity:
|
||||||
|
|
||||||
|
| Category | Items Found | Action Required |
|
||||||
|
|----------|-------------|------------------|
|
||||||
|
| Stored data | `golem15_fonoteka_user_collection_contexts` written by resolve/switch/provision. `river_job.args` will hold the raw invite token until River prunes completed jobs | Code. Decide the retention or encryption of the token in job args (Security, T-12-07) |
|
||||||
|
| Live service config | The Typesense index `golem15_fonoteka_albums` keeps documents after a collection cascade unless per-row deletes fire | Code fix in `Collection.BeforeDelete` |
|
||||||
|
| OS-registered state | None. Verified: no cron/systemd entries in scope | None |
|
||||||
|
| Secrets/env vars | `app.key` (`key: ""` in fonoteka.go/config/app.yaml) feeds `CollectionFingerprint`. PHP HMACs the raw config string, e.g. `base64:...` | Code. Tests use a test-only key. `collection_key` is masked by tide (`if key == "collection_key"`) [VERIFIED: tide/normalize.go:57-66] |
|
||||||
|
| Build artifacts | None | None |
|
||||||
|
|
||||||
|
## Common Pitfalls
|
||||||
|
|
||||||
|
### Pitfall 1: The token group's group-level `inv.scope:read`
|
||||||
|
**What goes wrong:** the Go token group is `surf.Use("inv_token", "throttle:fonoteka-api-token", "inv.scope:read")` [VERIFIED: routes.go:48]. Adding `inv.scope:write` per route makes write routes require **both** scopes. PHP routes carry exactly one scope each [VERIFIED: routes.php:454-517], so a write-only token passes in PHP and gets 403 in Go.
|
||||||
|
**How to avoid:** move the scope to each route (`g.Get("/genres", h, "inv.scope:read")`) and extend the route-table test (D-10) to assert exactly one `inv.scope:*` per token route.
|
||||||
|
|
||||||
|
### Pitfall 2: `Where` constrains only the immediately preceding route
|
||||||
|
**What goes wrong:** `g.Where("id", "[0-9]+")` attaches to the last registered route [VERIFIED: surf/router.go:267-284]. Routes like `albums/{id}/photos/{fileId}` need two `Where` calls right after that route.
|
||||||
|
**How to avoid:** a route-table test asserting that every `{id}`, `{fileId}` and `{collectionId}` has a constraint.
|
||||||
|
|
||||||
|
### Pitfall 3: Laravel validation semantics
|
||||||
|
**What goes wrong:**
|
||||||
|
- Laravel stops validating an attribute after an implicit rule (`required`) fails. Non-implicit rules don't run on empty or absent values.
|
||||||
|
- The size message depends on the value type (`max.string` / `max.array` / `max.numeric` / `max.file`).
|
||||||
|
- Wildcard attributes appear as `albums.0.name`.
|
||||||
|
- Messages come in the request locale.
|
||||||
|
|
||||||
|
The recorded example is `{"error":"Validation failed","errors":{"albums":["Pole albums jest wymagane."]}}` for `{"albums":[]}`: one message, even though `min:1` would also fail [VERIFIED: fixtures/routes/POST___fonoteka_api_v1_albums_bulk_jwt.yaml]. `after_or_equal` and `before_or_equal` are absent from Winter's pl catalog [VERIFIED: grep of validation.php], so they fall back to English.
|
||||||
|
**How to avoid:** one request-validator implementation with a table-driven test per recorded 422. Port the full pl/en catalogs.
|
||||||
|
|
||||||
|
### Pitfall 4: PHP `(int)` casts and `$request->boolean()`
|
||||||
|
`(int) $request->get('per_page', 20)` takes the leading numeric prefix (`"12abc"` → 12, `"abc"` → 0 → clamped to 1). `$request->boolean()` is true for `1, "1", true, "true", "on", "yes"`. Port these as named helpers (`phpInt`, `laravelBoolean`) with tests.
|
||||||
|
|
||||||
|
### Pitfall 5: Ordering differences (SQLite recording vs Postgres replay vs MariaDB production)
|
||||||
|
- `GET albums` uses `orderBy('name')` with no collation and no tiebreaker [VERIFIED: AlbumApiController.php:52].
|
||||||
|
- `GET styles` uses `orderBy('name')` [VERIFIED: StyleApiController.php:193].
|
||||||
|
- `GET collections` sorts in PHP: `->get()->sortBy('name')`, a stable byte comparison in PHP 8 [VERIFIED: CollectionApiController.php:44-50].
|
||||||
|
- Stats genres use `orderByDesc(album_count)`, with ties left unspecified.
|
||||||
|
|
||||||
|
**How to avoid:** use `pl-x-icu` where production MariaDB `utf8mb4_polish_ci` would apply, always add `id ASC` as a tiebreaker, sort collections in Go by name bytes with a stable sort over id-ordered rows, and keep recording data free of collation-sensitive collisions.
|
||||||
|
|
||||||
|
### Pitfall 6: The album-created notification side effect
|
||||||
|
**What goes wrong:** PHP's `Plugin::registerNotificationListeners` hooks `eloquent.created` on Album and calls `NotificationService::notifyAlbumAdded`. That writes a `golem15_fonoteka_notifications` row for each collection member except the actor, and publishes `notification:new` and `notification:count` to `user:{id}` [VERIFIED: Plugin.php:87-111; NotificationService.php:52-80, 265-283]. Album create and bulk in a shared collection change DB state and Centrifugo traffic.
|
||||||
|
**How to avoid:** port `write()` + `notifyAlbumAdded` now, next to `notifyInvitationAccepted` (D-14). The wishlist branch is Phase 13. Publish through `lighthouse.Emit` on the write tx, not synchronously as PHP does.
|
||||||
|
|
||||||
|
### Pitfall 7: Invitation email validation differs
|
||||||
|
`$request->validate(['email' => 'required|email'])` uses Laravel's `email` rule, then `normalizeEmail` applies `FILTER_VALIDATE_EMAIL` after `strtolower(trim())` and throws its own ValidationException [VERIFIED: InvitationApiController.php:63; InvitationService.php:249-257]. go-playground's `email` tag accepts a different set. Record the boundary cases and pin them in tests.
|
||||||
|
|
||||||
|
### Pitfall 8: Raw token persistence and logging
|
||||||
|
The raw invite token sits in `river_job.args` (a JSON column) until River prunes the row. River's default completed-job retention is about 24 h [ASSUMED]. Anyone with DB read access (or a backup) during that window can accept the invitation as the invitee, provided they also have a session for the invitee's email (accept checks `email === user.email`). That limits the impact.
|
||||||
|
**How to avoid:** encrypt the token in the args with the app key (`lagoon.Encrypted` exists [VERIFIED: lagoon/encrypted.go]) or delete or scrub the job row on completion. Never log the args.
|
||||||
|
|
||||||
|
### Pitfall 9: Commit-ordering of side effects
|
||||||
|
PHP sends invite mail and runs `notifyInvitationAccepted` after the transaction returns [VERIFIED: InvitationService.php:74, 178-180]. In Go, enqueue inside the tx (D-13) so a rollback drops them. Album notifications in bulk happen **inside** PHP's transaction (synchronous publish before commit). Go publishing after commit is the intended improvement (P11 D-06) and invisible to recorded bodies.
|
||||||
|
|
||||||
|
### Pitfall 10: `cover_urls` recordings need deterministic inputs
|
||||||
|
CoverImporter fetches from `discogs.com` / `*.discogs.com` over the network [VERIFIED: CoverImporter.php:206-226]. Record only the network-free failure cases (non-https, off-allow-list host), which yield `cover_import_failures: [url]`. Cover the success path with Go tests through an injectable fetch func.
|
||||||
|
|
||||||
|
**Mapping note:** PHP's `cover_host_suffix` `.discogs.com` maps to fetchguard `AllowHosts: ["discogs.com"]` (dotted-suffix match) [VERIFIED: fetchguard/fetch.go:131-143].
|
||||||
|
|
||||||
|
### Pitfall 11: WebP decode and thumbs
|
||||||
|
Go's stdlib registers no webp decoder. `image.DecodeConfig` (the `getimagesize` stand-in) and `Thumb` fail for `.webp` unless `golang.org/x/image/webp` is imported, and imaging cannot encode webp. `defaultEncodeImage` falls back to JPEG bytes in a `.webp`-named thumb [VERIFIED: attach/thumb.go:92-104]. Decide this explicitly: register the decoder and accept JPEG-in-webp thumbs, or restrict.
|
||||||
|
|
||||||
|
## Code Examples
|
||||||
|
|
||||||
|
### Share token generation (port of CollectionShareService::generateToken)
|
||||||
|
```go
|
||||||
|
// Source: CollectionShareService.php:35-63 (TOKEN_LENGTH = 16, ALPHABET, REJECT_AT = 248)
|
||||||
|
const shareAlphabet = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
|
||||||
|
func generateShareToken() (string, error) {
|
||||||
|
out := make([]byte, 0, 16)
|
||||||
|
buf := make([]byte, 16)
|
||||||
|
for len(out) < 16 {
|
||||||
|
if _, err := rand.Read(buf); err != nil { return "", err }
|
||||||
|
for _, b := range buf {
|
||||||
|
if len(out) == 16 { break }
|
||||||
|
if b >= 248 { continue }
|
||||||
|
out = append(out, shareAlphabet[int(b)%62])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return string(out), nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
The share envelope `stateFor` is `{enabled, name, token, path ("/k/"|"/w/"+token), generated_at}` [VERIFIED: CollectionShareService.php:177-192]. Mutations lock the row (`FOR UPDATE`) and force-fill the share columns.
|
||||||
|
|
||||||
|
### Collection fingerprint
|
||||||
|
```go
|
||||||
|
// Source: CollectionFingerprint.php:301 — substr(hash_hmac('sha256', 'fonoteka:collection:' . $id, (string) config('app.key')), 0, 32)
|
||||||
|
func CollectionKey(appKey string, id uint) string {
|
||||||
|
m := hmac.New(sha256.New, []byte(appKey)) // raw config string, including any "base64:" prefix
|
||||||
|
m.Write([]byte("fonoteka:collection:" + strconv.FormatUint(uint64(id), 10)))
|
||||||
|
return hex.EncodeToString(m.Sum(nil))[:32]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### PHP-verbatim discrete values to reuse
|
||||||
|
- Album search text fields [VERIFIED: AlbumSearchService.php:43-48]: `'query_by' => 'name,artist_display,style_names,genre_name,track_titles,notes,label,catalog_number'`, `'query_by_weights' => '10,10,5,5,3,1,3,3'`, `'sql_like' => ['name', 'notes', 'artist_display', 'track_titles', 'label', 'catalog_number']`, `'sql_like_relations' => ['artists.name']`.
|
||||||
|
- `private const SORTS = ['name', 'created_at', 'rating', 'year', 'artist', 'price'];` and `RATING_FILTERS = ['5', '4+', '3+', '2+', '1+', 'none']` [VERIFIED: AlbumApiController.php:41-43].
|
||||||
|
- `AlbumSyncService::DEFAULT_PER_PAGE = 100`, `MAX_PER_PAGE = 200`. The cursor is `base64_encode($syncVersion->toIso8601String() . '|' . $id)` [VERIFIED: AlbumSyncService.php].
|
||||||
|
- Completeness `BASE_TAGS = ['cover', 'year', 'cat', 'label', 'genre', 'tracks', 'country']`, plus `CONDITION_TAG = 'cond'`, `PRICE_TAG = 'price'`, `RATING_TAG = 'rating'` [VERIFIED: AlbumCompletenessService.php:15-37].
|
||||||
|
- Invitation token `bin2hex(random_bytes(32))`, `hash('sha256', $rawToken)`, `now()->addDays(7)`. Mail paths `'/en/invitations/' . $rawToken` / `'/zaproszenia/' . $rawToken`. Views `golem15.fonoteka::mail.collection_invitation-en` / `golem15.fonoteka::mail.collection_invitation` [VERIFIED: InvitationService.php:59-65, 263-279].
|
||||||
|
- Manual cover error messages per reason (`invalid_url`, `scheme`, `unresolvable`, `private_ip`, `network_error`, `http_status`, `content_type`, `too_large`, `invalid_image`) [VERIFIED: AlbumApiController.php:623-637].
|
||||||
|
- `MediumFamily` map `"LP": "vinyl", "2LP": "vinyl", "EP 7\"": "vinyl", "CD": "cd", "2CD": "cd", "MC": "cassette", "Box": "box"` [VERIFIED: models/album_search.go].
|
||||||
|
|
||||||
|
## State of the Art
|
||||||
|
|
||||||
|
| Old Approach | Current Approach | When Changed | Impact |
|
||||||
|
|--------------|------------------|--------------|--------|
|
||||||
|
| Scout total = engine `found` | Scout recounts in SQL when a query callback is set | Scout 10.x (installed v10.25.0) | Finding 1: the Go total must recount |
|
||||||
|
| Go stub serializers (P5) | The full `SerializesFonoteka` port | This phase | The created/updated broadcast goldens flip to assertions |
|
||||||
|
|
||||||
|
**Deprecated/outdated:** the "inflated total quirk" wording in D-16. The `genres` seed hook can be retired once POST genres is ported, since register/login were ported in P7 [CITED: parity/README.md].
|
||||||
|
|
||||||
|
## Assumptions Log
|
||||||
|
|
||||||
|
| # | Claim | Section | Risk if Wrong |
|
||||||
|
|---|-------|---------|---------------|
|
||||||
|
| A1 | River's default completed-job retention is about 24 h, so the raw token stays in `river_job.args` for about a day | Pitfall 8 | Longer or shorter exposure window. Mitigation advised either way. |
|
||||||
|
| A2 | PHP `resolve()`'s fallback without a kind filter can pick a wishlist when its id is lower. The Go `kind=collection` fallback may diverge | Pattern 1 | Wrong active collection for users whose wishlist predates their collection. Record a case. |
|
||||||
|
| A3 | Winter renders HttpException 410/409 under APP_DEBUG=false as the production error page with that status | Finding 3 | Wrong embedded bytes. Re-recording settles it. |
|
||||||
|
| A4 | The ValidationException envelope from `$request->validate()` on these routes is Laravel's `{"message","errors"}`, or HTML | Finding 3 | 422 body mismatch. Must record. |
|
||||||
|
| A5 | The publication normalizer does not mask the album subtree's `created_at`/`updated_at` | Finding 5 | If it already does, there is less tide work |
|
||||||
|
| A6 | `isSiteAdmin` can be false for everyone in Phase 12 because the Go user plugin has no `users_groups` | Open Q3 | `can_manage_org`/`can_use_ai`/`can_import_discogs` are wrong for site admins |
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None of these is decided yet. Each needs the user's confirmation at the plan-count checkpoint.
|
||||||
|
|
||||||
|
1. **D-16 correction (Finding 1).**
|
||||||
|
- What we know: installed Scout v10.25.0 recounts `total` in SQL over min(found, 1000) re-gated ids.
|
||||||
|
- What's unclear: whether the user keeps D-16's wording ("total from Typesense") or accepts the correction.
|
||||||
|
- Recommendation: implement the Scout recount ("mirror Scout exactly") and reword D-16 step 3 and success criterion 3. The D-18 leak test then also asserts that `meta.total` excludes poisoned ids.
|
||||||
|
2. **Port `realtime/channels` here? (Finding 2)**
|
||||||
|
- What we know: `me/context` has no channel. `realtime/channels` returns `collection:<id>`, is pending in the manifest, and the Nuxt app calls it.
|
||||||
|
- Recommendation: yes. It goes in the collections plan on the JWT group only, and API-01's wording maps onto `me/context` plus `realtime/channels`.
|
||||||
|
3. **Site-admin predicate.**
|
||||||
|
- What we know: `OrgAccess::isSiteAdmin` reads `$user->groups` (code `admin`) through `users_groups` [VERIFIED: user/models/User.php:50]. The Go user plugin has neither the table nor the relation [VERIFIED: grep].
|
||||||
|
- Options: (a) false for everyone in Phase 12, with a documented gap for Phase 13 credentials; (b) add the Winter user-groups schema to the Go user plugin, which is a core-plugin contract change and needs the user's sign-off.
|
||||||
|
- Recommendation: (a).
|
||||||
|
4. **Photo URL prefix fix (Finding 4).**
|
||||||
|
- Recommendation: config only (`bucket_url: file://./storage/app/uploads/public`, `public_path_prefix: /storage/app/uploads/public`), plus an exported URL helper in attach and a regression check on avatar URLs.
|
||||||
|
5. **WebP (Pitfall 11).**
|
||||||
|
- Recommendation: register `golang.org/x/image/webp` (bump to v0.46.0 in summercms.go) and document that webp thumbs are JPEG bytes. The alternative is to accept that webp uploads fail the decode check, which would diverge from PHP.
|
||||||
|
6. **Raw token in job args (Pitfall 8).**
|
||||||
|
- Recommendation: store the token encrypted with the app key in the job args (`lagoon.Encrypted`), and never log the args. Scrubbing the River row after send is the fallback.
|
||||||
|
|
||||||
|
## Environment Availability
|
||||||
|
|
||||||
|
| Dependency | Required By | Available | Version | Fallback |
|
||||||
|
|------------|------------|-----------|---------|----------|
|
||||||
|
| Go toolchain | everything | ✓ | go1.27.0 | — |
|
||||||
|
| Docker (testcontainers Postgres) | DB tests, parity replay | ✓ | 29.7.2 | — (TestMain fails without it, by design) |
|
||||||
|
| PHP CLI (isolated parity instance) | New recordings (D-08) | ✓ | 8.5.10 | — |
|
||||||
|
| Node | `capture_clients.mjs` (Nuxt capture) | ✓ | v22.23.2 | Hand-scripted `tide` flows via `parity:record --spec` |
|
||||||
|
| Typesense server | — | not needed | — | Fake engine (D-18), D-12 records with search off |
|
||||||
|
| Centrifugo | — | not needed | — | tide fake recorder on 127.0.0.1:8424, memory driver |
|
||||||
|
|
||||||
|
**Missing dependencies with no fallback:** none.
|
||||||
|
|
||||||
|
## Validation Architecture
|
||||||
|
|
||||||
|
### Test Framework
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| Framework | Go `testing` (+ testify assert/require, go1.27 fuzzing), testcontainers Postgres |
|
||||||
|
| Config file | none. `parity/parity_test.go` TestMain starts Postgres. |
|
||||||
|
| Quick run command | `cd fonoteka.go && go test ./plugins/golem15/fonoteka/... -short -count=1` |
|
||||||
|
| Full suite command | `cd fonoteka.go && go vet ./... && go test ./... -count=1`, plus `cd summercms.go && go test ./... -count=1` for the framework changes |
|
||||||
|
| Parity | `cd fonoteka.go && go test ./parity -run 'TestParityCorpus|TestBroadcastGoldens|TestNuxtFlow' -count=1` |
|
||||||
|
|
||||||
|
### Phase Requirements → Test Map
|
||||||
|
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||||||
|
|--------|----------|-----------|-------------------|-------------|
|
||||||
|
| API-01 | Collections CRUD/photos/image/switch/me-context/realtime-channels/share replays | parity | `go test ./parity -run TestParityCorpus/.*collection` | ❌ fixtures for new cases (Wave 0) |
|
||||||
|
| API-01 | nuxt-collections flow (invite → accept → members → remove) | parity flow | `go test ./parity -run TestNuxtFlow/nuxt-collections` | ❌ |
|
||||||
|
| API-01 | Token pin, switch refusal for tokens, pending-invitation 409, provisioning | integration | `go test ./plugins/golem15/fonoteka/classes -run TestActiveCollection` | partial (`classes/postgres_test.go` harness) |
|
||||||
|
| API-01 | Invite mail is enqueued in the tx and absent on rollback; token never logged; locale/link per inviter | integration | `go test ./plugins/golem15/fonoteka/... -run TestInvitationMail` (postcard memory) | ❌ |
|
||||||
|
| API-02 | Album CRUD/rating/photos/bulk/stats/value/missing/sync/search replays (search off) | parity | `go test ./parity -run TestParityCorpus/.*albums` | ❌ new cases |
|
||||||
|
| API-02 | Broadcast goldens created/updated/deleted/bulk asserted | parity | `go test ./parity -run TestBroadcastGoldens` | ✅ (created/updated pending) |
|
||||||
|
| API-02 | Leak test D-18 (5 cases, plus `meta.total` excluding poison) | security | `go test ./plugins/golem15/fonoteka/... -run TestSearchLeak` | ❌ |
|
||||||
|
| API-02 | Upload guard, `system_files` row, blob, thumb, MaxBytes cap | integration | `go test ./plugins/golem15/fonoteka/... -run TestAlbumPhotoUpload` | ❌ |
|
||||||
|
| API-02 | Manual cover URL reasons (private_ip, scheme, …) and cover_urls allow-list | unit | `go test ./plugins/golem15/fonoteka/classes -run 'TestManualCover|TestCoverImporter'` | ❌ |
|
||||||
|
| C-02 | Request-DTO fuzz over every write endpoint (route-table enumerated) | fuzz | `go test ./plugins/golem15/fonoteka -run=^$ -fuzz=FuzzWriteEndpoints -fuzztime=60s` + seed corpus in `go test` | ❌ |
|
||||||
|
| D-10 | Twin scopes are exactly one `inv.scope`; JWT-only routes are absent from the token group | unit | `go test ./plugins/golem15/fonoteka -run TestRouteTable` | ✅ extend `routes_group_test.go` / `routes_isolation_test.go` |
|
||||||
|
| Framework | beachcomber found/weights, tide multipart + url mask, validator semantics + catalogs | unit | `cd summercms.go && go test ./modules/beachcomber/... ./modules/tide/... ./modules/lagoon/... ./modules/phrasebook/...` | ❌ |
|
||||||
|
|
||||||
|
### Sampling Rate
|
||||||
|
- **Per task commit:** the quick run command, plus `go vet`.
|
||||||
|
- **Per wave merge:** the full suite in both repos, plus the parity run.
|
||||||
|
- **Phase gate:** full suite green in both repos, parity corpus with the new routes flipped to `ported` and passing, `check_corpus.go --require-recorded --check-secrets` green, and `TestDocsTree` green for the framework README changes.
|
||||||
|
|
||||||
|
### Wave 0 Gaps
|
||||||
|
- [ ] Re-record the HttpException cases under APP_DEBUG=false (accept 410, switch 404, household/members 404, invitations 404, the token 404).
|
||||||
|
- [ ] tide request `body_file` (multipart) + `url`/`thumb_url` disk-name normalizer + publication date masking (framework).
|
||||||
|
- [ ] Seed hooks or flows for a second user (bob) and an outsider. `id:outsider` exists from P11; reuse it.
|
||||||
|
- [ ] The fake `beachcomber` engine with scripted ids and `found` for D-18.
|
||||||
|
|
||||||
|
## Security Domain
|
||||||
|
|
||||||
|
### Applicable ASVS Categories
|
||||||
|
|
||||||
|
| ASVS Category | Applies | Standard Control |
|
||||||
|
|---------------|---------|-----------------|
|
||||||
|
| V2 Authentication | yes (inherited) | `jwt.auth`, `inv_token` guard (P6/P8), unchanged |
|
||||||
|
| V3 Session Management | no | Stateless bearer |
|
||||||
|
| V4 Access Control | yes | `AccessibleBy` (membership + token narrowing), owner checks for destroy/image/share/household, token-group isolation, 404-never-403 |
|
||||||
|
| V5 Input Validation | yes | The Laravel-semantics request validator, the Fill allow-lists (P5), the request-DTO fuzz |
|
||||||
|
| V6 Cryptography | yes | `crypto/rand` tokens, sha256 at rest, HMAC fingerprint, `subtle.ConstantTimeCompare` where comparing secrets |
|
||||||
|
| V7 Error/Logging | yes | No raw invite or share tokens in logs. Opaque 500s. Winter error pages without debug detail. |
|
||||||
|
| V11 Business Logic | yes | Inline throttles (switch, regenerate, invite, resend 10/min; photos 20/min; sync 60/min) |
|
||||||
|
| V12 Files/Resources | yes | ImageContentGuard (sniff + decode), body caps, fetchguard SSRF |
|
||||||
|
| V13 API | yes | Per-route `inv.scope`, JWT-only routes absent from the token group |
|
||||||
|
|
||||||
|
### Known Threat Patterns (candidate T-12-xx, each with a failing-when-broken test)
|
||||||
|
|
||||||
|
| ID | Pattern | STRIDE | Standard Mitigation |
|
||||||
|
|----|---------|--------|---------------------|
|
||||||
|
| T-12-01 | Cross-collection IDOR on `albums/{id}`, `collections/{id}`, photos/{fileId} | Information disclosure / Tampering | One `AccessibleBy` scope plus the `collection_id` re-gate. The file lookup is scoped to the owner's relation. Same 404 body for foreign and missing ids. |
|
||||||
|
| T-12-02 | Search leak through stale or mis-scoped index docs, including `meta.total` | Information disclosure | SQL re-gate on items and the total (D-18 + Finding 1) |
|
||||||
|
| T-12-03 | A personal token escaping its pinned collection (search, stats, value, sync, artists, genres) | Elevation of privilege | Token pin in `Resolve` + `AccessibleBy` narrowing. `switch` returns 404 for tokens. |
|
||||||
|
| T-12-04 | A token reaching owner-only interactive surfaces (share, household, invitations, switch, sync, me/context) | Elevation of privilege | Absent from the token group (route-table test) plus the in-handler `personalTokenIsActive` second lock |
|
||||||
|
| T-12-05 | An editor publishing or deleting someone else's collection | Elevation of privilege | Explicit `owner_id == user` checks for share, image, destroy and household |
|
||||||
|
| T-12-06 | Invitation token reuse, expiry or email mismatch | Spoofing | `FOR UPDATE` lookup by hash, `isPending`, email equality → 410. Resend rotates the hash. |
|
||||||
|
| T-12-07 | Raw invite token at rest (river_job.args) or in logs | Information disclosure | Encrypt the args or scrub after send. No logging of args. `summer_jobs` not used. |
|
||||||
|
| T-12-08 | Share token predictability | Spoofing | `crypto/rand` + rejection sampling, unique-index loop, `FOR UPDATE` mutation |
|
||||||
|
| T-12-09 | SSRF through `cover_url` / `cover_urls` | Tampering / Information disclosure | fetchguard PublicOnly (manual) / AllowHosts discogs.com (import), no redirects, byte cap, timeouts |
|
||||||
|
| T-12-10 | Malicious upload (polyglot, oversized) | Tampering / DoS | `body.limit` upload bytes, a 10240 KB rule, magic-byte sniff + decode check, throttle 20/min (JWT) |
|
||||||
|
| T-12-11 | Mass assignment (`owner_id`, `collection_id`, `public_token`, `kind`, `market_price_source`) | Tampering | Fill allow-lists + the request-DTO fuzz (C-02) |
|
||||||
|
| T-12-12 | Context race causing double provisioning | Tampering | `users` row `FOR UPDATE` before the context row (PHP lock order) + the unique `user_id` context key |
|
||||||
|
| T-12-13 | `realtime/channels` raw id treated as a tenant selector | Elevation of privilege | Never re-accepted on write paths. The collection authorizer re-validates membership on subscribe. |
|
||||||
|
|
||||||
|
## Recommended Plan Split (for the plan-count checkpoint)
|
||||||
|
|
||||||
|
Lean, five plans. Recordings go into each feature plan. The unit tests are last.
|
||||||
|
|
||||||
|
1. **12-01 Framework gaps (summercms.go)**
|
||||||
|
- beachcomber: an optional `found` + `QueryByWeights`.
|
||||||
|
- tide: multipart request bodies, a disk-name URL normalizer, album date masking in publication goldens.
|
||||||
|
- A Laravel-semantics request validator with the full pl/en catalogs. Fold `lagoon-validate-min-message`.
|
||||||
|
- attach: export the original-file URL.
|
||||||
|
- The webp decoder decision.
|
||||||
|
- READMEs and docs updated.
|
||||||
|
2. **12-02 Active context, collections and share (fonoteka.go)**
|
||||||
|
- The token-aware `Resolve`/`SwitchTo`, provisioners, `AccessibleBy`, gates, `CollectionFingerprint`.
|
||||||
|
- The full `Serialize*` port and the photo URL prefix fix.
|
||||||
|
- Collections CRUD/photos/image/switch, `me/context`, `realtime/channels`, `collection/share` with show, update and regenerate.
|
||||||
|
- Token-group restructure with per-route scopes, the `Collection.BeforeDelete` per-row fix, and recordings for these routes.
|
||||||
|
3. **12-03 Household and invitations**
|
||||||
|
- `InvitationService` and `OrgProvisioner`.
|
||||||
|
- `NotificationService.write`, `notifyInvitationAccepted` and `notifyAlbumAdded`.
|
||||||
|
- The invitation mail job (encrypted args) and postcard templates.
|
||||||
|
- Invitation, member and accept routes, the `nuxt-collections` flow recording and the HTML error pages.
|
||||||
|
4. **12-04 Albums, search and lookups**
|
||||||
|
- The full AlbumWriteService (artists, styles, tracklist text, dates, duplicate matcher, completeness, CoverImporter).
|
||||||
|
- ImageContentGuard and the manual cover fetcher.
|
||||||
|
- Album CRUD, bulk, rating, photos, stats, value, missing and sync.
|
||||||
|
- AlbumSearchService with the Scout-exact recount.
|
||||||
|
- Artists, styles and genres POST.
|
||||||
|
- The `nuxt-albums` flow, the upload replays, and flipping the broadcast goldens.
|
||||||
|
5. **12-05 Unit tests and security**
|
||||||
|
- The request-DTO fuzz over route-table write endpoints.
|
||||||
|
- The D-18 leak test with total assertions.
|
||||||
|
- Route-table scope and constraint tests.
|
||||||
|
- Validator, parser and completeness unit tests.
|
||||||
|
- Coverage for everything above, plus the T-12-xx tests.
|
||||||
|
|
||||||
|
**Alternative (4 plans):** merge 12-03 into 12-02. That plan gets very large, so it is not recommended. Waves: 12-01 → 12-02 → 12-03 → 12-04 → 12-05. Run them sequentially: 12-03 and 12-04 both edit `routes.go`, `serialize.go` and the manifest.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
### Primary (HIGH confidence, read this session)
|
||||||
|
- `/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/routes.php` (the full file).
|
||||||
|
- PHP controllers: CollectionApiController, CollectionShareController, MeContextController, InvitationApiController, CollectionMemberController, AlbumApiController, AlbumSyncController, RatingApiController, ArtistApiController, StyleApiController, GenreApiController (store).
|
||||||
|
- PHP classes: ActiveCollectionResolver, CollectionProvisioner, OrgProvisioner, OrgAccess, AiGate, DiscogsGate, DiscogsConfigResolver, CollectionFingerprint, CollectionShareService, InvitationService, NotificationService (write paths), AlbumWriteService, ArtistResolver, AlbumSearchService, AlbumSyncService, AlbumCompletenessService, AlbumDuplicateMatcher, TracklistTextParser, AddedDateParser, ImageContentGuard, ManualCoverUrlFetcher, discogs/CoverImporter, PolishOrder.
|
||||||
|
- PHP traits and models: SerializesFonoteka, models Collection/Album/CollectionInvitation, Plugin.php notification listener.
|
||||||
|
- PHP vendor: `laravel/scout` v10.25.0 Builder.php, Engines/TypesenseEngine.php, EngineManager.php, Searchable.php; `winter/storm` Http Kernel; `modules/system/models/File.php`; `modules/system/lang/pl/validation.php`; `config/cms.php`.
|
||||||
|
- Go app: routes.go, classes/{active_collection,album_write_service,collection_write_service,artist_resolver,serialize}.go, models/{album,album_search,collection,collection_invitation,notification,api_token}.go, middleware/token_scope.go, classes/auth/token_guard.go, search.go, realtime.go, controllers/genre_controller.go, user avatar upload, config/{app,http,storage}.yaml.
|
||||||
|
- Go framework: beachcomber (searchable.go, sync.go, typesense/engine.go), lighthouse (Emit, WithoutBroadcasting), conga (Dispatch, Enqueue, Job), fetchguard, lagoon (validate.go, paginate.go, attach/*), surf/router.go, tide (flow.go, normalize.go), phrasebook catalogs.
|
||||||
|
- Parity corpus: manifest.yaml, README.md, php_parity.sh, capture-rules.yaml, fixtures (routes, broadcasts, nuxt).
|
||||||
|
- Planning: 11-CONTEXT.md decisions, 05-CONTEXT D-14..D-19, PITFALLS.md Pitfall 15, pending todos.
|
||||||
|
|
||||||
|
### Secondary
|
||||||
|
- `go list -m golang.org/x/image@latest` → v0.46.0 (Go module proxy).
|
||||||
|
|
||||||
|
### Tertiary (LOW)
|
||||||
|
- River completed-job retention default (A1). Training knowledge, not verified this session.
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
**Confidence breakdown:**
|
||||||
|
- PHP contract and helpers: HIGH. Read line by line.
|
||||||
|
- The Scout total semantics: HIGH. Read from the installed vendor source.
|
||||||
|
- Go seams and gaps: HIGH. Read from the source.
|
||||||
|
- Error-page and ValidationException envelopes: MEDIUM. Need re-recording.
|
||||||
|
- Plan sizing: MEDIUM.
|
||||||
|
|
||||||
|
**Research date:** 2026-10-02
|
||||||
|
**Valid until:** 2026-11-01 (stable: PHP reference frozen, framework under active development)
|
||||||
Reference in New Issue
Block a user