Files
summercms/.planning/phases/14-domain-jobs-and-external-integrations/14-RESEARCH.md
2026-10-03 18:56:43 +02:00

97 KiB
Raw Blame History

Phase 14: Domain jobs and external integrations - Research

Researched: 2026-10-03 Domain: Go port of Płytarium's outbound integrations (Discogs, Anthropic/OpenAI-compatible AI, G15Office), River job workers, console commands, and two new shared core plugins (sm-golem-plugin, sm-feedback-plugin) Confidence: HIGH for the PHP behaviour and the Go seams (all read this session); MEDIUM for the proposed new framework designs (fetchguard client, upstream sidecars); LOW only where marked [ASSUMED].

<user_constraints>

User Constraints (from CONTEXT.md)

Locked Decisions

AI layer (sm-golem-plugin)

  • D-01: The Go port of PHP Golem15.Golem lives in its own shared core-plugin repo, git@git.golem15.com:golem15/sm-golem-plugin.git (created empty by the user). Module git.golem15.com/golem15/sm-golem-plugin, mounted in fonoteka.go at plugins/golem15/golem as a git submodule, following .planning/notes/core-plugins-own-repos.md. The plugin ID stays golem15.golem, with the same settings key (golem15_golem_settings) and lang namespace, so migrated PHP data loads as-is. — Reversibility: costly — the plugin ID, settings key and module path are what every consuming application and the migrated settings row use.
  • D-02: The provider adapters (Anthropic Messages, OpenAI-compatible chat completions) are hand-rolled JSON mappings over the guarded outbound client (folded todo fetchguard-guarded-http-client). No Anthropic or OpenAI SDK dependency. PHP's adapters are raw curl, which makes request parity easier.
  • D-03: Port the Golem Settings model in full: the models repeater (name, adapter openai|anthropic, encrypted api_key, base_url, model, system_prompt, is_enabled, is_default, generates_images, accepts_images, has_files_endpoint, max_completion_tokens) plus its admin settings screen, reading the existing settings row. getVisionModel() fills fonoteka's classes.AdminVisionModel seam (Phase 13), so the admin tier of ResolveAIConfig and the site-admin branch of AIAllowed light up.
  • D-04: Port all of AIService except faces: send, sendStream, sendFile/sendFilePath (files endpoint), sendWithModel, sendToImageModel, sendToVisionModel, generateImage, ask, the model system-prompt application, Prompt/AIResponse value objects and PromptFactory. CompreFace/FaceService and ChatContextCollector are not ported. The SSRFGuard is ported as part of the guarded-client work.
  • D-05: Base-URL guard: user and org credential base_url overrides always go through the private/reserved-IP dial guard. Admin-configured Settings models are trusted (they may point at a LAN/local endpoint such as Ollama). The researcher confirms this split against PHP's SSRFGuard and reports any difference before planning.
  • D-06: REQUIREMENTS INTG-02 ("Anthropic Go SDK") is reworded at plan time to "Anthropic and OpenAI-compatible adapters over the guarded client". PROJECT.md's repository paragraph gains sm-golem-plugin and sm-feedback-plugin, and drops feedback and sitemap from the application-plugin list.

Route scope

  • D-07: All 6 album Discogs/AI routes that Phase 12 D-04 moved here are in scope (albums/match, albums/{id}/match, albums/{id}/apply-release, albums/recognize on the JWT and token groups, albums/import/discogs), together with the routes the criteria already name. ROADMAP SC4/SC5 are reworded at plan time to list them. They share ReleaseMatchScorer, AlbumReleaseApplicator, DiscogsImportResolver, DiscogsMapper and AlbumRecognitionService with the named routes.
  • D-08: The Phase 13 ReleaseFetcher seam behind the CSV row-edit selected_discogs_id pick gets the real Discogs getRelease + DiscogsMapper::mapRelease implementation, so a pick resolves exactly as in PHP.
  • D-09: GET/DELETE oauth-identities (social login, deferred since Phase 7) and GET /api/v1/fonoteka/me (the minimal endpoint from Phase 8 D-20) are not part of Phase 14. They stay pending and are flagged for the roadmap (see Deferred).
  • D-10: WR-02 from the Phase 13 review is fixed before the CSV workers go live: the mapping, row-edit and cancel writes lock the import row or compare-and-swap on its status, as commit already does, so none of them can race a commit and queue a second import job. Response shapes stay the same. This is a deliberate deviation from PHP, which has the same race.
  • D-11: The researcher checks routes.php and the Nuxt/MCP callers for any recorded case that is still missing, for example a successful selected_discogs_id pick, the cover-price route, and the recognize success and truncation paths. It records those against the PHP backend, together with their upstream exchanges (D-15).

Feedback plugin (sm-feedback-plugin)

  • D-12: Feedback is a shared core plugin in its own repo, git@git.golem15.com:golem15/sm-feedback-plugin.git (created by the user). Module git.golem15.com/golem15/sm-feedback-plugin, mounted in fonoteka.go at plugins/golem15/feedback, plugin ID golem15.feedback, with the same tables (feedback_submissions, user preferences) and the same settings key. — Reversibility: costly — the plugin ID, tables and settings key become the contract other applications mount.
  • D-13: Full port. It covers GET /_feedback/api/v1/{key}/config (throttle 60/min/IP), POST {key}/submit (multipart, 10/min/IP, Origin allowlist gate, ImageContentGuard), the OPTIONS {any} 204 preflight, and JWT PUT me/hidden. embed.js is served at the same /plugins/golem15/feedback/assets/js/embed.js path that the Nuxt proxy expects. The plugin also ports the Settings model (enabled, widget_key, allow_hide, position, allowed_origins, colors, pl/en labels) with its admin screen, the submissions admin list, and the golem15.user.getApiArray hook that adds feedback_widget_hidden to the user payload. The user plugin already references this field. SyncFeedbackToG15Office becomes a River job on the guarded client (JSON + multipart). Feedback routes are recorded as new parity fixtures. API-08 is reworded to feedback only.
  • D-14: Sitemap is dropped for Płytarium. Nuxt builds its own sitemap with @nuxtjs/seo, and fonoteka registers no menu item types. A todo records a 1:1 sitemap plugin port for the next blog project (grzybyfunkcjonalne.pl or golem15.com).

Vendor testing

  • D-15: Upstream HTTP in tests: the upstream Discogs/Anthropic/OpenAI/G15Office exchanges are recorded once as a sidecar to each parity case and served from an httptest fake that the guarded client points at during replay. Replay is deterministic and offline. The fake also asserts the request Go sends upstream (method, path, headers including User-Agent and auth, body) against PHP's recorded request. CI makes no live vendor calls, and G15Office is verified only through the fake.
  • D-16: Time is injected. The Discogs rate limiter, its wait budget and retry-after handling, and the match job's re-enqueue all take a clock/sleeper interface. Tests advance a fake clock without real sleeps, and assert the 240 s timeout and the re-enqueue delay as values.
  • D-17: The researcher reads DiscogsRateLimiter.php and decides where its state lives, either process memory or Postgres (an UNLOGGED table or an advisory lock), depending on whether PHP's cross-worker guarantee is still needed when one binary runs HTTP and River workers. The choice and its reason go in RESEARCH.md.

Folded Todos

  • fetchguard-guarded-http-client (.planning/todos/pending/fetchguard-guarded-http-client.md, high): extend fetchguard from a guarded HTTPS GET into a guarded outbound http.Client/constructor (POST, PUT, multipart upload, bearer auth), keeping the dial-time private/reserved-IP rejection, with an explicit trusted mode for admin-configured endpoints (D-05). It replaces Apparatus RequestSender. Its consumers are the Discogs client, the Golem adapters and feedback's G15OfficeClient. Framework change in summercms.go: update modules/fetchguard/README.md and docs/.
  • redacting-slog-handler (.planning/todos/pending/redacting-slog-handler.md, medium): a framework slog.Handler wrapper porting Apparatus RedactCredentialsTap. It redacts the keys api_key, apikey, authorization, bearer, password, secret, token, webhook_secret and admin_password (case-insensitive, nested groups) and scrubs messages with the PHP regex patterns. It also checks whether surf's error path already gives SafeExceptionResponse behaviour. This is the first phase to send user credentials to outside services.

Claude's Discretion

  • Wishlist digest mail content, locale and template: a straight port of WishlistDigestJob and its PHP mail view on the existing postcard mail pipeline.
  • prune-notifications and reindex options, output and exit codes: a straight port of PruneNotifications.php and ReindexAlbums.php, on bonfire with the existing schedule.go entry.
  • Exact package layout inside sm-golem-plugin and sm-feedback-plugin, which follows .planning/notes/plugin-layout-winter-directories.md.
  • Plan split, subject to the lean-mode plan-count checkpoint.

Deferred Ideas (OUT OF SCOPE)

  • Sitemap plugin 1:1 port for the next blog project: a new todo is written alongside this context.
  • oauth-identities GET/DELETE (social login) and GET /api/v1/fonoteka/me: the routes stay pending without a phase. The roadmap needs a home for them before Phase 15, whose criterion is all routes green. A new todo is written.
  • Golem face detection (CompreFace/FaceService) and ChatContextCollector: not ported until a consumer needs them.

Reviewed Todos (not folded)

  • backend-admin-api-tokens, bonfire-duplicate-command-names, lagoon-readme-after-commit-callback-order, nest-framework-packages-under-modules, per-module-readmes-after-nest, readme-go-fences-src, refresh-fonoteka-readme, rewrite-summercms-readme, scaffold-* (3), wristband-neutral-resource-default, 2026-10-01-benchmark-…: these matched on keywords only and are unrelated to the phase domain. </user_constraints>

<phase_requirements>

Phase Requirements

ID Description (REQUIREMENTS.md, before the D-06/D-13/D-14 rewording) Research Support
JOBS-02 CSV import write job and Discogs match job (240 s timeout) are ported; the match job re-enqueues itself with a delay on a Discogs rate-limit error instead of failing the batch §"The three PHP jobs", §D-17, §"WR-02 fix", Pattern 3 (job worker), Pitfalls 4-7
JOBS-03 Wishlist digest coalesces notifications in a 30-minute window and deletes its queue row on completion §"The three PHP jobs" (WishlistDigestJob), Pattern 3
SRCH-02 The reindex command asserts zero documents with collection_id 0 before and after, and can drop the legacy index §"Console commands" (ReindexAlbums), beachcomber gap (DropIndex)
INTG-01 Discogs client with proactive rate threshold, bounded in-request wait budget, retry-after fallback, and host-locked cover fetch, plus the cover-price route, the wishlist match/apply-release routes, discogs-credential/test and the CSV row-edit pick §"Discogs client", §D-17, §D-11 case list, §"Inbound limiters"
INTG-02 AI cover recognition through provider adapters (reword: Anthropic and OpenAI-compatible adapters over the guarded client) with per-credential model and base URL overrides, plus ai-credential/test and the backend global vision model §"Golem plugin surface", §D-05, §"Settings storage gap"
API-08 Feedback submissions and sitemap output (reword: feedback only, D-13/D-14) §"Feedback plugin surface"
CLI-05 Płytarium commands: oauth-client (already shipped Phase 8), prune-notifications, reindex with drop-old-index flag §"Console commands"
</phase_requirements>

Project Constraints (from CLAUDE.md)

From summercms.go/CLAUDE.md, fonoteka.go/CLAUDE.md, the parent CLAUDE.md and the user's global CLAUDE.md:

  • Lean planning: few, large plans; a plan-count checkpoint before PLAN.md files are written; unit tests are always the last plan of the phase.
  • Go conventions: standard library first; add a dependency only when the research doc or a phase decision names it. go vet and go test ./... green at every commit. D-02 forbids an Anthropic/OpenAI SDK: this phase adds no new third-party Go dependency.
  • Compiled plugins only; no runtime plugin loading.
  • API parity is the acceptance test. Do not "improve" response shapes. D-10 (WR-02) is the one sanctioned deviation and keeps response shapes.
  • Commits: never add co-author tags; one logical change per commit; planning docs and code in separate commits.
  • Framework docs rules: any change to a modules/ package's exported API, config keys, CLI commands or dependencies updates that module's README.md and the affected docs/ pages in the same change; a new module ships a README with the standard structure and a row in the root README modules table; framework READMEs never name a consuming application; every identifier named in a README/docs page must exist (go test ./cmd/summer -run TestDocsTree, summer docs:build --check). Config keys in docs are checked by hand.
  • Two repositories: summercms.go is framework only and knows nothing about Płytarium; app code goes to fonoteka.go. Planning docs stay in summercms.go/.planning.
  • Core plugin contracts: user, blog, pages, payment PHP plugins are shared; Go ports preserve contracts; PHP originals are not changed. sm-user-plugin changes must be additive.
  • Submodule workflow (fonoteka.go/CLAUDE.md, core-plugins-own-repos.md): change a core plugin inside its checkout, commit (and push) there first, then commit the bumped pointer in fonoteka.go as a separate commit; never stage submodule files from the app repo. ssu is available for submodule management.
  • GSD workflow enforcement: file changes go through GSD commands.

Summary

Phase 14 is the largest port so far by PHP volume: about 2,250 lines of Discogs classes, 900 lines of Discogs/AI controllers, 650 lines of jobs, 450 lines of AlbumRecognitionService, 900 lines of the Golem AI layer and 1,650 lines of the feedback plugin (including embed.js). Almost everything it needs from the framework already exists: conga (River) with Dispatch/Delay/Timeout/StopJob/CompleteJob, the bonfire command pattern, beachcomber's Typesense engine, lagoon file attachments, surf.MemoryStore for in-controller rate limiters, the user plugin's GetApiArrayEvent, and the Phase 13 seams (job_contract.go kinds and args, ReleaseFetcher, AdminVisionModel, ResolveAIConfig, DiscogsAllowed). The framework gaps are concrete and small: fetchguard is GET-only with test hooks that are unexported, cabana settings cannot express a repeater, beachcomber cannot report whether a dropped index existed, and tide has no notion of upstream-exchange sidecars.

The three researcher tasks resolve as follows. D-05: the admin-trusted / user-and-org-guarded split matches PHP, but PHP's SSRFGuard::assertSafeUrl is stricter and different in kind: https only, a host allowlist (default only .openai.com and the DALL-E blob host, env-overridable), and a resolve-time private-IP check. Its failure is an uncaught RuntimeException, so PHP answers with the Winter 500 HTML page, not {ok:false}. The Go port must keep the allowlist and the 500 to stay parity-true, and add the dial-time guard on top. D-17: the limiter state goes in Postgres (an UNLOGGED table and one atomic upsert), because conga explicitly supports a separate queue:work process (queue.work_in_serve: false) and the 50/60-per-minute budget is shared by every process. D-11: every currently recorded Phase 14 case is a negative path (404, 503 disabled, 403, 422, a live-vendor 401). All success, rate-limit, truncation and pick cases are still missing, and the two live-vendor cases must be re-recorded with a sidecar.

Two findings contradict CONTEXT and need user confirmation before planning (see Open Questions). First, the PHP Golem settings code is golem_settings, not golem15_golem_settings. Second, the PHP models[].api_key is stored in plaintext, not encrypted. The project's settings convention (Phase 5, user-resolved) is a dedicated typed table, not Winter system_settings, and cabana has no repeater field type. So "reading the existing settings row" needs a storage decision (recommended: a golem15_golem_models table, edited as an admin list/form, plus an import from the PHP JSON at cutover).

Primary recommendation: Build the framework pieces first (guarded fetchguard.Client with a trusted mode and a test transport seam, the redacting slog handler, tide upstream sidecars plus a recording proxy, beachcomber DropIndex). Then port in vertical slices: Discogs core and jobs, Discogs routes, sm-golem-plugin with AI recognition, sm-feedback-plugin. Finish with a unit-test plan and a check-phase14.sh gate.

Architectural Responsibility Map

Capability Primary Tier Secondary Tier Rationale
Guarded outbound HTTP (POST/PUT/multipart/bearer, dial-time IP guard, trusted mode) Framework (summercms.go fetchguard) — Shared by Discogs, Golem and G15Office; SSRF policy is a framework concern
Credential redaction in logs Framework (new slog handler) App wiring (slog.SetDefault in serve/queue:work) Nothing publishes a *slog.Logger today; every module falls back to slog.Default()
Upstream exchange sidecars (record + replay fake + request assertion) Framework (tide) App (parity/ harness wiring) tide owns fixtures; the app owns which routes use sidecars
Discogs client, limiter, mapper, scorer, applicator, resolvers App (fonoteka plugin classes/discogs) DB (UNLOGGED limiter table) Płytarium-specific domain
Discogs/AI/credential-test routes App (fonoteka controllers/api, routes.go) — API parity surface
CSV match/import, digest workers App (fonoteka plugin jobs.go or jobs/) Framework (conga/River) Kinds and args are fixed by job_contract.go
AI providers, Prompt/AIResponse, model settings Core plugin (sm-golem-plugin) Framework (fetchguard) Shared across Golem15 apps (D-01)
Feedback widget API, settings, G15Office sync Core plugin (sm-feedback-plugin) User plugin event (GetApiArrayEvent) Shared core plugin (D-12)
fonoteka:prune-notifications, fonoteka:reindex App (fonoteka console/) Framework (bonfire, conga schedule, beachcomber) Straight ports

Standard Stack

Core (all already in the build; nothing new to install)

Library / module Version Purpose Why standard here
Go stdlib net/http, mime/multipart, encoding/json, crypto/hmac, crypto/sha256, log/slog Go 1.27.0 (go version this session) Adapters, multipart upload, HMAC bucket id, redaction handler D-02: hand-rolled JSON over the guarded client [VERIFIED: go version]
modules/fetchguard in-repo Base of the guarded client Existing dial-time classifier (ip.go) is a strict superset of PHP's lists [VERIFIED: modules/fetchguard/ip.go:5-20]
modules/conga (River v0.47.0) in-repo Workers, Delay re-dispatch, Timeout, record rows conga.Timeout, DispatchOpts.Delay, StopJob, CompleteJob exist [VERIFIED: modules/conga/job.go:21-37, conga.go:112-133]
modules/beachcomber + beachcomber/typesense in-repo Reindex: Upsert/Flush/SearchPage Engine has Upsert, Delete, Flush, SearchIDs; PageSearcher gives Found [VERIFIED: modules/beachcomber/searchable.go:39-57]
modules/surf MemoryStore in-repo In-controller limiters (fonoteka-discogs-missing:, fonoteka-recognize:, fonoteka-discogs-import:) Attempt(key, max, decay) is atomic check+increment and returns retryAfter [VERIFIED: modules/surf/limiter_store.go:11-13]
modules/lagoon/attach in-repo Feedback screenshot attachOne attach.Relation{Name, Many, Public} [VERIFIED: modules/lagoon/attach/relation.go]
modules/postcard in-repo Digest mail templates Existing mail jobs use postcard.Mailer.Send [VERIFIED: fonoteka plugin jobs.go:119-126]
modules/cabana in-repo Admin settings and list screens Settings are scalar-only singleton rows (see gap) [VERIFIED: modules/cabana/settings.go:37-95]

Supporting

Library Version Purpose When to Use
testcontainers-go postgres v0.44.0 (already in go.mod) Limiter SQL, worker and migration tests Postgres-dependent tests, -short skippable
pgx/v5 v5.10.0 (already in go.mod) — Unchanged

Alternatives Considered

Instead of Could Use Tradeoff
Hand-rolled adapters anthropic-sdk-go / openai-go Locked out by D-02
Postgres limiter table Process-memory limiter (like PubfailCounter) Breaks the cross-process budget when queue.work_in_serve: false (see D-17)
MITM recording proxy for sidecars Hand-written sidecars from vendor docs Hand-written sidecars cannot capture what PHP actually sent; D-15 asks for PHP's recorded request

Installation: none. go.mod changes are limited to the two new submodule modules (require plus local replace) and their own go.mod files.

Package Legitimacy Audit

This phase installs no external packages (D-02 rules out vendor SDKs; every capability is stdlib or in-repo). The seam check was not needed.

Package Registry Age Downloads Source Repo Verdict Disposition
(none) — — — — — —

Packages removed due to [SLOP] verdict: none Packages flagged as suspicious [SUS]: none

Existing Go seams and enqueue sites (inventory for the planner)

Seam / file What exists Phase 14 action
fonoteka/classes/job_contract.go:29-62 CsvImportKind = "golem15.fonoteka.csv_import", CsvImportQueue = "fonoteka_csv_import", CsvImportLabel = "fonoteka.csv.import", CsvMatchKind = "golem15.fonoteka.csv_match", CsvMatchQueue = "fonoteka_csv_match", CsvMatchLabel = "fonoteka.csv.match", WishlistDigestKind = "golem15.fonoteka.wishlist_digest", WishlistDigestJobQueue = "fonoteka_wishlist_digest", WishlistDigestLabel = "wishlist_digest", WishlistDigestDelay = 1800 * time.Second; args CsvImportArgs{CsvImportID uint \json:"csv_import_id"`}, CsvMatchArgs{CsvImportID}, WishlistDigestArgs{SubscriberID `json:"subscriber_id"`, WishlistCollectionID `json:"wishlist_collection_id"`}` [VERIFIED: job_contract.go:29-88] Register workers with exactly these kinds and queues. Never rename (queued rows carry them)
fonoteka/jobs.go:32-41 Jobs() returns invitation mail and purchase mail on conga.OnQueue(...), conga.MaxAttempts(...); comment says "The CSV and wishlist digest kinds have no worker until Phase 14 (D-04)" Add three conga.Job(...) entries. Update the comment and TestJobContract* (they assert the queues are unserved)
classes/csv_import_service.go:519-587 UpdateCsvMapping Dispatches CsvMatchArgs on CsvMatchQueue, label, Count: RowCount, metadata {"csv_import_id":N} WR-02 lock (below)
csv_import_service.go:589-628 ReleaseFetcher + SetReleaseFetcher (atomic box) Default always returns ErrDiscogsUnavailable Install the real fetcher at plugin Boot (D-08). ErrDiscogsRateLimited maps to discogs_rate_limited 422
csv_import_service.go:639-705 UpdateCsvRow Pick path: digits check, candidate allow-list, DiscogsAllowed, fetch, resolved WR-02 lock. Success now writes the draft
csv_import_service.go:825-882 CommitCsvImport CAS UPDATE ... WHERE id = ? AND status = ? preview→importing, then Dispatch CsvImportArgs Unchanged (reference CAS)
csv_import_service.go:889-901 CancelCsvImport Cancels MatchJobID/ImportJobID from the stale read WR-02 lock
classes/wishlist_notifications.go:130-153 EnqueueWishlistDigest Upsert golem15_fonoteka_wishlist_digest_queue with RETURNING (xmax = 0), Dispatch WishlistDigestArgs with Delay: WishlistDigestDelay only on first insert Worker only
classes/ai_config_resolver.go:40-45 var AdminVisionModel = func(ctx context.Context) (*AIConfig, error) { return nil, nil }; AIConfig{Adapter, APIKey, BaseURL, Model} all json:"-"; comment: "the outbound client that consumes it (Phase 14) owns the SSRF guard PHP applies" Replace from the golem plugin at Boot. Add a trust marker (for example Trusted bool \json:"-"``) so the admin tier selects trusted mode (D-05)
classes/gates.go:60-90 AIAllowed Admin branch calls AdminVisionModel(ctx) Lights up automatically
classes/gates.go:136 ResolveDiscogsConfig, :177 DiscogsAllowed, :188 MarketCurrency Ported tiers Reuse in the client forUser
classes/cover_importer.go:34-94 Fetch seam; fetchguard.Fetch with AllowHostsMode Reuse for the host-locked cover fetch in AlbumCoverFetcher and the applicator
classes/manual_cover_fetcher.go:84 Fetch func(ctx, rawURL, policy) seam pattern Pattern precedent for injectable fetchers
classes/public_share.go PubfailCounter In-memory per-process limiter with an injected clock Precedent: in-request inbound limiters stay in memory
fonoteka/search.go:19-47 settingsGate reads search_use_typesense; wireSearch installs the gate Reuse in reindex's "configured" check
fonoteka/schedule.go:13-17 {Command: "fonoteka:prune-notifications", Cadence: pact.Daily()} Only the command is missing
fonoteka/console/oauth_client.go bonfire.Command{Name, Description, Args, Flags (Bare), Run} pattern Template for the two new commands
controllers/albums_admin_controller.go:57-90 Phase 10.1 admin discogsLookup/discogsSync stubs, documented as "Phase 14 replaces" Not in CONTEXT; see Open Question 5
routes_table_phase13_test.go:270-282 phase14-routes-absent Asserts the Phase 14 routes are not mounted Update as routes land
sm-user-plugin controllers/api_controller.go:795-806 "feedback_widget_hidden": false default, then app.Events.Collect(ctx, &classes.GetApiArrayEvent{User}) merges listener keys Feedback plugin listens (fonoteka's own listener at plugin.go:93 is the template)
fonoteka/plugins/golem15/fonoteka/config/config.yaml:22-36 discogs.token, market_currency, max_covers, cover_max_bytes, cover_timeout_seconds, cover_host_suffix Missing: user_agent, rate_threshold, wait_budget_seconds, retry_after_fallback_seconds. Add them under golem15.fonoteka.discogs.*. Keep base_uri a code constant (PHP comment: "literal, never env()/user input")
fonoteka.go/config/http.yaml CORS paths already include _feedback/api/* Nothing to add

The three PHP jobs (port targets)

AlbumCsvMatchJob (jobs/AlbumCsvMatchJob.php) [VERIFIED: read this session]

  • public int $timeout = 240; (line 36), so the Go job gets conga.Timeout(240 * time.Second). private const MAX_CANDIDATES = 10; (line 38).
  • Flow: import gone → failJob {"error":"import_not_found"}. checkIfCanceled → cancelJob (Go: StopJob). isTerminalOrSuperseded (status in canceled/done/importing/preview/failed, or matching with a different match_job_id) → completeJob {"skipped": "<status string>"}. Then status=matching, startJob(total=row_count), updateJobState(alreadyDone) when resuming, per pending row (ordered by row_index): cancel check → matchRow → updateJobState(done). End: status preview, error_message=null, completeJob {"matched": done}.
  • matchRow: duplicate in the collection → matched_csv, candidates [], matched_album_id. Discogs gate off → matched_csv, []. Else searchByQuery("artist title"), then searchByBarcode when empty. 0 results → matched, []. 1 result → getRelease + mapRelease into draft_json, candidates [mapSearchResult], matched. More → the first 10 mapSearchResult, draft_json=null, matched_ambiguous.
  • DiscogsRateLimitException → pauseAndReschedule: delay = max(e.retryAfterSeconds, limiter.forToken(token).secondsUntilAvailable()), pending count, status stays matching, Dispatch a new match job (label fonoteka.csv.match, count pending, metadata {csv_import_id, resumed_after_rate_limit: true, retry_after: delay}, delay), write match_job_id = next, then completeJob {"paused":"discogs_rate_limited","retry_after":delay,"next_job_id":next}. The $rescheduling static guard exists only for Laravel's sync driver; Go does not need it.
  • Any other throwable → status failed, error_message, failJob {"error": msg} and return normally (no retry). In Go, call FailJob and return nil, or River retries up to 3 times.

AlbumCsvImportJob (jobs/AlbumCsvImportJob.php) [VERIFIED]

  • No Discogs network. importerCanWrite (Collection::accessibleBy(user)->whereKey(collection_id)) before start and per row; on failure status canceled + cancelJob. Rows not written/skipped, ordered by row_index; startJob(count).
  • Per row inside Album::withoutBroadcasting: canonical rows (matched_csv with raw id) → CsvCanonicalIdMatcher::match, then applyCsvOverwrite/applyCsvFill or createCsv. Other rows → inputForRow (draft wins over CSV-mapped fields when there is a draft and the row is selected/matched/resolved; genre → genre_id via resolveGenreId), writeOptions (allow_null_format, first cover URL + import_covers), matcher->find → fillEmpty or create, syncCsvRating. Per-row exception → row error, error_code=write_failed.
  • End: status done. When writtenCount > 0, publish collection:{id} event collection.bulk_updated {"reason":"csv_import","count":N}. completeJob {"written": done}. Outer throwable → failed + failJob.
  • Go gap: album_write_service.go has CreateAlbum/UpdateAlbum/SaveAlbum but no CSV variants. Port fillEmpty, applyCsvFill, createCsv, applyCsvOverwrite, resolveGenreId, syncCsvRating and create($options) (PHP AlbumWriteService.php:64-373). classes/csv/canonical_id.go only has CanonicalID(row); the scoped matcher query (CsvCanonicalIdMatcher::match) also needs a port. Bulk-broadcast suppression exists in lighthouse (Phase 11 SC4).

WishlistDigestJob (jobs/WishlistDigestJob.php, 49 lines) [VERIFIED]

  • Read queue row (user_id, collection_id). Missing or item_count <= 0 → completeJob {"skipped": true}. Else delete the row, then NotificationService::mailWishlistDigest(subscriber, wishlist, count), then completeJob {"sent": count}.
  • mailWishlistDigest (NotificationService.php:204-232): wishlist must exist with kind === 'wishlist' and subscriber must exist. Locale en when preferred_locale === 'en', else pl. View golem15.fonoteka::mail.wishlist_subscription_digest(-en). Vars ownerName (wishlist owner name), itemCount, wishlistName. To: subscriber email. Copy views/mail/wishlist_subscription_digest.htm and -en.htm (absent in Go views/mail) and register them in mail.go.
  • Order matters: PHP deletes the row before sending. A failed send after the delete loses that digest. Keep the order for parity, and put the delete and the CompleteJob in one transaction where possible.

Discogs client (INTG-01)

DiscogsClient.php [VERIFIED]: TIMEOUT_SECONDS = 10. Headers Authorization: Discogs token=<token>, User-Agent: config('fonoteka.discogs.user_agent'), Accept: application/json. URL = rtrim(base_uri,'/') . path, with ids cast to int in the path and free text only in the query. Endpoints: getRelease(id) → /releases/{id}?curr_abbr=<marketCurrency>; getPriceSuggestions(id) → /marketplace/price_suggestions/{id} (an empty object is normal); getIdentity() → /oauth/identity; getMasterVersions(id) → /masters/{id}/versions?per_page=10; searchByBarcode → /database/search?barcode=&type=release; searchByQuery → /database/search?q=&type=release.

Request loop: limiter->acquire() once, then loop. Each response → syncFromHeaders(X-Discogs-Ratelimit-Remaining). 200 → json; 404 → null; 401/403 → DiscogsTokenRejectedException; 429 → wait = registerRetryAfter(Retry-After); if wait > remaining budget throw DiscogsRateLimitException(max(1, wait)), else sleep and retry. Any other status → ApplicationException("Discogs request failed with HTTP status N."). Logs carry only status and path.

Config values [VERIFIED: plugins/golem15/fonoteka/config/fonoteka.php:18-30]: 'base_uri' => 'https://api.discogs.com', 'user_agent' => env('DISCOGS_USER_AGENT', 'FonotekaApp/1.0 +https://github.com/golem15com/wn-fonoteka-plugin'), 'rate_threshold' => 50, 'wait_budget_seconds' => 15, 'retry_after_fallback_seconds' => 10.

Discogs itself throttles by source IP: 60/min authenticated, 25/min unauthenticated, as a 60 s moving average, with X-Discogs-Ratelimit, -Used and -Remaining headers [CITED: Discogs developer docs via search summary; the docs page returned 403 to direct fetch, LOW].

Domain classes to port alongside (PHP line counts): DiscogsMapper (434: mapRelease, mapSearchResult, mapMasterVersion), ReleaseMatchScorer (210, depends on completeness; Go completeness.go exists), AlbumReleaseApplicator (178, apply(album, mapped, overwriteAll, coverOnly, dryRun), uses CoverImporter), DiscogsImportResolver (209), DiscogsInputParser (133, static parse, normalizeBarcode, alternateBarcode), PriceSuggestionResolver (88), AlbumCoverFetcher (575). The PHP suite has unit tests for every one of these (tests/unit/Discogs*Test.php, ReleaseMatchScorerTest.php, PriceSuggestionResolverTest.php, AlbumReleaseApplicator*Test.php, CoverImporterTest.php, DiscogsRateLimiterTest.php, DiscogsClientTest.php). Use them as test vectors. The pure classes (DiscogsMapper, DiscogsInputParser, ReleaseMatchScorer, PriceSuggestionResolver) suit PHP-generated truth tables, the precedent being parity/csv_truth_tables.php, which runs the PHP classes directly and writes testdata/php_*.json.

Inbound (per-route) limiters — separate from the outbound Discogs limiter

Key (PHP) Limit Response when over Routes
fonoteka-discogs-missing: + user id or IP [VERIFIED: AlbumReleaseMatchController.php:31] tooManyAttempts(key, 60), hit(key, 60) 429 {"result":"error","code":"too_many_requests","retry_after":availableIn} albums match/apply (3 routes) and wishlist match/apply (same key, shared on purpose)
fonoteka-discogs-import: + id [VERIFIED: DiscogsImportController.php:68] 20 per 60 s 429 {"result":"error","code":"too_many_requests"} (no retry_after) albums/import/discogs (checked after validation)
fonoteka-recognize: + id [VERIFIED: RecognizeApiController.php:53-57] 10 per 60 s 429 {"error":"Too many requests"} recognize, both groups (after AiGate, before validation)
route throttle:12,1 12/min surf throttle 429 token cover-price

Use one surf.MemoryStore per plugin (process memory, precedent PubfailCounter). Attempt matches Laravel's tooManyAttempts+hit pair, and its retryAfter is availableIn. Order of checks per route is parity-visible. Copy it exactly (match/apply: album scope 404 → gate 503 → limiter → validation; import: gate → validation → limiter; recognize: AiGate 403 → limiter → validation → image guard).

D-17: where the Discogs limiter state lives — Postgres, UNLOGGED table, single atomic statement

PHP behaviour [VERIFIED: DiscogsRateLimiter.php:23-233]: a fixed window (WINDOW_SECONDS = 60) per bucket. Bucket id = substr(hash_hmac('sha256', token, app.key), 0, 32), never logged. Cache keys discogs:ratelimit:<bucket>:{count,window_start,lock}. Every read-modify-write runs under Cache::lock(..., 5)->block(3, ...) "regardless of driver" so that two near-simultaneous workers cannot both pass. acquire() loops: when a slot is free it increments and returns. Otherwise, if the wait exceeds the remaining wait budget (wait_budget_seconds, 15) it throws DiscogsRateLimitException(max(1, windowRemainder)), else it sleeps. secondsUntilAvailable() is read-only (1 when free). syncFromHeaders() only tightens: implied = max(0, threshold - Remaining) raises count when a window is active. registerRetryAfter() returns the numeric Retry-After or the fallback (10).

Decision: Postgres. Reasons:

  1. The cross-process guarantee is still needed. conga supports queue.work_in_serve: false with a separate fonoteka queue:work process (systemd), which is documented in the app's config/queue.yaml and the conga README [VERIFIED: fonoteka.go/config/queue.yaml; modules/conga/README.md "Configuration"]. Rolling restarts also briefly run two binaries. A process-memory limiter would give each process its own 50/min. Discogs throttles by source IP at 60/min, so two processes on one host could overshoot by up to 40/min, and the CSV match job would degrade to 429 handling.
  2. The cost is negligible: one short statement per Discogs request, which itself takes about 100-500 ms and is capped at 50/min per token.
  3. UNLOGGED gives cache semantics (no WAL, emptied after a crash), the same as PHP's file/redis cache. A lost window only means one fresh window.
  4. One atomic INSERT ... ON CONFLICT DO UPDATE ... WHERE ... RETURNING replaces PHP's lock. Postgres guarantees an atomic insert-or-update under concurrency, and a row "locked but not updated because an ON CONFLICT DO UPDATE ... WHERE condition was not satisfied ... will not be returned" [CITED: postgresql.org/docs/current/sql-insert.html]. So "a row returned" means "slot granted", with no advisory lock and no explicit transaction.
  5. Clock injection (D-16) still works: pass now from the injected clock as a parameter instead of SQL now(), so tests drive the window with a fake clock. The unit tests of the waiting logic use an in-memory RateStore fake. One testcontainers test proves the SQL.

Shape (migration in the fonoteka plugin; the table is Go-only, so add it to parity/schema_diff_test.go's Go-only allow-list as golem15_fonoteka_settings was) [ASSUMED design]:

CREATE UNLOGGED TABLE golem15_fonoteka_discogs_rate_windows (
  bucket       TEXT PRIMARY KEY,          -- 32-hex HMAC id, never the token
  window_start TIMESTAMPTZ NOT NULL,
  hits         INTEGER NOT NULL
);
-- tryAcquire($1 bucket, $2 now, $3 threshold): a returned row = granted
INSERT INTO golem15_fonoteka_discogs_rate_windows AS w (bucket, window_start, hits)
VALUES ($1, $2, 1)
ON CONFLICT (bucket) DO UPDATE SET
  window_start = CASE WHEN $2 - w.window_start >= interval '60 seconds' THEN $2 ELSE w.window_start END,
  hits         = CASE WHEN $2 - w.window_start >= interval '60 seconds' THEN 1 ELSE w.hits + 1 END
WHERE $2 - w.window_start >= interval '60 seconds' OR w.hits < $3
RETURNING hits;
-- not granted: SELECT window_start → wait = max(1, 60 - floor(now - window_start))
-- syncFromHeaders: UPDATE ... SET hits = GREATEST(hits, $implied) WHERE bucket = $1 AND $now - window_start < interval '60 seconds'

Keep the HMAC key as app.key (the bucket id is not shared with PHP, so byte-compatibility does not matter, but secrecy does). The exception's retryAfterSeconds is the window remainder, not the leftover budget, because the CSV match job uses it as the re-dispatch delay.

D-05: base-URL guard split, confirmed against PHP with differences

PHP SSRFGuard::assertSafeUrl [VERIFIED: plugins/golem15/golem/classes/security/SSRFGuard.php:40-67, config/ssrf.php:16-20]:

  1. parse_url must give scheme and host, else RuntimeException('Invalid URL').
  2. Scheme must be https (Only https:// scheme allowed).
  3. Host allowlist config('golem15.golem::ssrf.allowed_hosts'). A leading . is a suffix match; otherwise exact, case-insensitive. Default: 'oaidalleapiprodscus.blob.core.windows.net' and '.openai.com', overridable by GOLEM15_SSRF_ALLOWED_HOSTS (comma list). Failure: Host not in allowlist: <host>.
  4. Resolve A/AAAA (fallback gethostbynamel). No answer → Cannot resolve host. Any private IP → Resolves to private/loopback IP (127/8, 10/8, 172.16/12, 192.168/16, 169.254/16, 0/8, ::1, fe80::/10, fc00::/7, plus filter_var NO_PRIV_RANGE|NO_RES_RANGE).

Callers [VERIFIED by grep + read]: UserAiConfig::fromCredential (line 43), OrgAiConfig::fromOrg (line 42) and AiCredentialController::resolveTestConfig (line 150, inline body base_url). All of them apply it only when a base_url override is present; the default provider URLs are never checked. Plugin.php:82 also guards the AI-returned DALL-E image URL before File::fromUrl. Admin Golem Settings models go through AIService::send → RequestSender::sendPostRequest, which has no URL validation (validateUrl runs only on GET/download) [VERIFIED: apparatus/classes/RequestSender.php:37-57, 123-169].

The split matches D-05 (admin trusted, user and org guarded). Differences to carry into the plan:

# PHP D-05 as written Recommendation
1 User/org guard is https-only + host allowlist + resolve-time IP check. The allowlist is the primary defence (the class comment says so) "private/reserved-IP dial guard" only Port the allowlist (config key golem15.golem.ssrf.allowed_hosts in sm-golem-plugin, same default, env override) and keep the dial-time guard. Without the allowlist Go accepts https://api.groq.com, which PHP rejects: a parity and security regression
2 With the default allowlist, any user/org Claude credential with an explicit base_url (even https://api.anthropic.com/v1) is rejected, because .anthropic.com is not listed. Unknown whether production sets GOLEM15_SSRF_ALLOWED_HOSTS (.env is secret-guarded) — Open Question 1
3 Guard failure throws RuntimeException. AiCredentialController::test catches only ApplicationException, and recognize calls the resolver outside its try, so PHP answers the Winter 500 HTML page (the recorded POST ai-credential __bad-base-url fixture shows that page for store) — Go must return the same 500 page (controllers/api/winter_500.html exists) on guard failure, not {ok:false}. Record cases (D-11 list)
4 PHP's IP check is resolve-time (DNS-rebind TOCTOU is acknowledged in the class doc) dial-time Go's dial-time check is strictly stronger and invisible to parity
5 RequestSender POST sets CURLOPT_FOLLOWLOCATION and no timeout — Guarded mode: never follow redirects (or re-guard each hop). Trusted mode: follow is acceptable. Set an explicit AI timeout of 120 s (PHP's sendStream uses CURLOPT_TIMEOUT 120) [ASSUMED value for non-stream]
6 generateImage returns a URL that callers must assertSafeUrl before fetching — Port AssertSafeURL as an exported helper in sm-golem-plugin. Fetch any AI-returned URL only via fetchguard AllowHosts mode with the same allowlist

D-11: missing parity cases and their upstream sidecars

All 14 pending routes already have one fixture each, and every one is a negative path recorded with DISCOGS_TOKEN= empty and no AI configured [VERIFIED: parity/manifest.yaml pending entries; fixture bodies read]:

Route (pending) Recorded today Notes
POST wishlist/albums/{id}/match jwt 404 {"error":"Album not found"}
POST wishlist/albums/{id}/apply-release jwt 404
POST albums/match jwt 503 discogs_disabled
POST albums/{id}/match jwt 404
POST albums/{id}/apply-release jwt 404
POST albums/recognize jwt 403 {"error":"AI features not available"} Manifest says status: 422, the fixture says 403. Fix the manifest when flipping to ported
POST albums/import/discogs jwt 503 discogs_disabled request body {"query":...} (the real field is input)
POST ai-credential/test jwt 200 {"ok":false,"error":"Incorrect API key provided: sk-parit******real. ..."} Live OpenAI call, no sidecar. Re-record
POST discogs-credential/test jwt 200 {"ok":false,"error":"Token Discogs jest nieprawidłowy lub wygasł."} Live Discogs 401, no sidecar. Re-record
POST /api/v1/fonoteka/albums/recognize token 422 photo required
POST /api/v1/fonoteka/albums/{id}/cover-price/discogs token 200 {"fetched":false,"reason":"no_source"}
PATCH import/csv/{id}/rows/{rowId} (ported route) discogs-off 422 exists; no success pick

Cases to record against PHP (each with an upstream sidecar unless marked "none"), from routes.php, the Nuxt store (app/stores/fonoteka.ts:315-440, 601-620, 1011, 1156) and the MCP client (fonoteka-mcp/src/client.ts:277-337):

# Route Case Upstream sidecar
1 PATCH import/csv/{id}/rows/{rowId} pick success → 200 resolved + draft (Nuxt sends {selected_discogs_id: "<id>"}) Discogs GET /releases/{id}?curr_abbr=EUR 200
2 same pick rate-limited → 422 discogs_rate_limited Discogs 429 (+/- Retry-After)
3 same release 404 → 422 discogs_unavailable Discogs 404
4 POST albums/{id}/match 200 candidates with q, year, year_delta, medium /database/search?q= 200
5 same 422 validation (missing q) none (stops before upstream; needs a Discogs-enabled user)
6 same 429 too_many_requests inbound (61st call) none, or 60 sidecar entries; prefer a Go unit test
7 same 502 discogs_token_rejected Discogs 401
8 same 429 discogs_rate_limited (budget exhausted) Discogs 429 with Retry-After > 15
9 POST albums/match 200 candidates against draft search 200
10 POST albums/{id}/apply-release 200 {data, filled, remaining, draft}, dry_run:true, cover_only:true, overwrite_all (Nuxt MatchReleaseDialog sends all three) /releases/{id} 200 + cover image GET (i.discogs.com)
11 same 200 discogs_no_match (release 404) Discogs 404
12 POST wishlist/albums/{id}/match, .../apply-release 200 success twins (no dry_run; response has no draft key) as 4 / 10
13 POST albums/import/discogs draft (release URL/id), candidates (barcode with many hits), no_match (barcode, zero hits: 200 with barcode), 422 validation, 429 inbound search and/or release, master versions
14 POST /api/v1/fonoteka/albums/{id}/cover-price/discogs fetched:true (discogs_id path, cover + price suggestion), nothing_missing, ambiguous, rate_limited, refresh_price:true (MCP sends {refresh_price:true} or {}) release, price_suggestions, cover GET
15 POST discogs-credential/test {token} body ok → {"ok":true}; stored token rejected (re-record); rate limited; disabled (no body, gate off: no upstream) /oauth/identity 200/401/429
16 POST ai-credential/test inline {provider:"claude", api_key, model} ok; inline OpenAI 401 (re-record the existing case); stored credential; unsafe base_url → 500 Winter page (no upstream); non-allowlisted host → 500; no credential → {"ok":false,"error":"No AI credential configured."} Anthropic POST /v1/messages, OpenAI POST /v1/chat/completions
17 POST albums/recognize jwt 200 albums (BYOK user), 200 {"albums":[],"code":"recognition_truncated"} (both responses non-JSON, retry stop_reason:max_tokens/finish_reason:length), 200 {"albums":[]} (unparseable twice, not truncated), 502 provider error, 422 non-image bytes (ImageContentGuard), 429 inbound, admin-tier with global vision model 1-2 AI exchanges per case
18 token albums/recognize 200 success with MCP multipart (photo filename cover, locale) AI exchange
19 feedback (new manifest section) GET {key}/config 200 / 404 bad key / 403 origin; POST {key}/submit 202 / 422 / 422 bad image / 403; OPTIONS 204; PUT me/hidden 200 / 422 none at request time. G15Office exchanges are job-side: record them as a job sidecar (create task + attachment) and assert from the River worker test
20 CSV flow match job paused on 429 and resumed; import job writing rows Discogs exchanges; job rows golden (summer_jobs)

How sidecars fit the existing layout [VERIFIED: parity/README.md, php_parity.sh, parity_test.go:233-297, fixtures/routes/*.yaml]:

  • Fixtures are tide flow YAML (version: 1, steps[] of request/response/normalize). Route cases live under fixtures/routes/<METHOD>_<path>_<group>__<case>.yaml, flows under fixtures/nuxt|mcp, and DB goldens next to them as *.rows.json (precedent: nuxt-csv.rows.json, compared in fonoteka_flows_test.go:107-155).
  • Proposed sidecar: <fixture>.upstream.yaml next to each fixture, holding an ordered list of exchanges {step, host, request:{method, path, query, headers (allow-listed: User-Agent, Accept, Content-Type, anthropic-version, anthropic-beta; auth headers masked to a placeholder), body (JSON-normalized; base64 image replaced by a sha256)}, response:{status, headers (Retry-After, X-Discogs-Ratelimit-*), body}}. tide loads it with the flow. replayPortedRoute serves it through an in-process fake (a RoundTripper or an httptest server) that asserts each Go request against the recorded one and fails on unconsumed or extra exchanges.
  • Recording PHP's side: PHP's Discogs base URI is a code literal ('base_uri' => 'https://api.discogs.com', no env), and the AI path is raw curl. So the only way to capture what PHP really sends, without editing PHP, is a recording HTTPS proxy. Guzzle honours HTTPS_PROXY in any SAPI and libcurl honours https_proxy/HTTPS_PROXY [ASSUMED: Guzzle/libcurl env-proxy behaviour]. Add a summer parity:upstream command to tide that (a) acts as a CONNECT proxy, (b) terminates TLS with a locally generated CA (stdlib crypto/x509), (c) in script mode answers from a hand-authored vendor-response file (deterministic, no real tokens), or in forward mode passes through to the real vendor once, and (d) writes the sidecar. Extend php_parity.sh serve with HTTPS_PROXY=http://127.0.0.1:8425 and -d curl.cainfo=<parity CA> (the Centrifugo recorder on 127.0.0.1:8424 is the precedent). Seed fake BYOK credentials (Discogs token matching ^[A-Za-z0-9_\-]{10,255}$, AI key sk-parity-…) in parity/fonoteka_reset.php. check_corpus --check-secrets must stay green, so mask the auth headers in sidecars.
  • If the proxy is judged too large, the fallback is: sidecars authored by hand from vendor docs, and the request assertion compares against a request derived from the PHP source rather than a recorded one. This weakens D-15 and needs user sign-off.

WR-02 fix (D-10)

[VERIFIED: 13-REVIEW.md:131-158; csv_import_service.go:519-587, 639-705, 889-901]. In one lagoon.Transaction, SELECT ... FOR UPDATE the import row (clause.Locking{Strength: "UPDATE"}), re-check csvBeforeCommit(cur.Status) on the locked row, then cancel cur.MatchJobID/cur.ImportJobID from the locked row, write, and dispatch. Apply this to UpdateCsvMapping, UpdateCsvRow (the save closure) and CancelCsvImport. UpdateCsvRow's Discogs fetch (up to 15 s wait budget) must happen outside the lock: fetch first, then lock, re-check status, and save. Otherwise a slow Discogs call holds the row lock and blocks commit. Add the review's interleaving test (commit vs mapping: at most one import job). Recommended, not mandated by D-10: make the match/import workers' own status writes (matching, preview, done, canceled) conditional on the locked row too. PHP's worker can otherwise resurrect a canceled import to matching. Response shapes are unaffected.

jobs.CancelJob runs on its own connection while the transaction holds the csv_imports row lock. It touches only summer_jobs and river_job rows, so no deadlock [ASSUMED: no FK or trigger from those tables back to csv_imports].

fetchguard: what the guarded client needs

Current API [VERIFIED: modules/fetchguard/fetch.go, policy.go]: Fetch(ctx, rawURL, Policy, *compass.Config) (*Result, error), GET only. Policy{Mode (AllowHostsMode|PublicOnlyMode), AllowHosts, MaxBytes, Timeout} plus unexported tlsConfig and skipReservedCheck test hooks, reachable only from the package's own tests (withTestLoopback). Redirects are never followed. Proxy: nil. The dial Control rejects reserved IPs (including NAT64/6to4-embedded v4 and zoned v6). Typed Error{Reason} with reasons invalid_url|scheme|unresolvable|private_ip|network_error|too_large.

Needed (design [ASSUMED], to be fixed in the plan):

  • fetchguard.NewClient(policy ClientPolicy, cfg *compass.Config) *Client (or *http.Client plus helpers) with Do(req), plus helpers PostJSON(ctx, url, headers, v), PutJSON, PostMultipart(ctx, url, headers, fields, file{field, name, mime, reader}), and a Bearer(token) header helper. Responses are capped at MaxBytes, typed errors are kept, and the status code is returned, not judged (PHP callers inspect bodies, not status).
  • Modes: AllowHostsMode, PublicOnlyMode, and TrustedMode (no dial guard, no host check, http allowed, for admin-configured endpoints such as LAN Ollama).
  • Test transport seam that production input cannot reach: for example fetchguard.WithTransport(ctx, rt) or a Client option set only by code, which the parity harness uses to route api.discogs.com/api.anthropic.com/api.openai.com/G15Office to the sidecar fake while keeping the original host in the asserted request. Production URLs stay code literals (Discogs base_uri).
  • Request bodies (a base64 photo of up to about 14 MB) are not capped by fetchguard. Set the timeout per consumer: Discogs 10 s (PHP TIMEOUT_SECONDS = 10), AI 120 s [ASSUMED], G15Office 30 s [ASSUMED; PHP has none].
  • Update modules/fetchguard/README.md, docs/services/outbound-http.md, plus mentions in docs/architecture/introduction.md and docs/setup/coming-from-wintercms.md (RequestSender is now covered) [VERIFIED: grep docs].

Redacting slog handler

PHP RedactCredentialsTap [VERIFIED: apparatus/classes/logging/RedactCredentialsTap.php]: keys api_key, apikey, authorization, bearer, password, secret, token, webhook_secret, admin_password, OPENAI_API_KEY, ANTHROPIC_API_KEY, PERPLEXITY_API_KEY (the todo omits the last three; include them, the comparison is case-insensitive). Values become [REDACTED] recursively; string values are also scrubbed with the patterns /Bearer\s+[A-Za-z0-9._\-+\/=]+/i → 'Bearer [REDACTED]', /sk-[A-Za-z0-9]{20,}/ → 'sk-[REDACTED]', /x-api-key:\s*[^\s,]+/i → 'x-api-key: [REDACTED]'. Go: a slog.Handler wrapper that rewrites the record message and every Attr (recursing into slog.KindGroup and handling WithAttrs/WithGroup). Nothing publishes a *slog.Logger today: beachcomber, lighthouse, postcard, conga and flare all do app.Lookup[*slog.Logger]() then fall back to slog.Default() [VERIFIED: grep]. So install it via slog.SetDefault(slog.New(redact.Wrap(base))) in the serve, queue:work and schedule:run entry points (or have backpack publish one). Host it in an existing module or a new small module; a new module needs a README and a root-table row. SafeExceptionResponse: surf already has recoverJSON/recoverBare panic recovery [VERIFIED: modules/surf/router.go:435-443, 638]. Whether it hides messages outside debug is unverified. A test in the plan should pin it.

Golem plugin surface (sm-golem-plugin)

[VERIFIED: plugins/golem15/golem/*, read this session]

  • AIService::send(Prompt, ?config): config defaults to Settings::getDefaultModel(). No config → failure('No AI model configured. Please add a model in Settings > AI.'). Empty key → failure('API key is not configured for the selected model.'). Model defaults to gpt-4o. applyModelSystemPrompt sets the model's system_prompt only when the prompt has none. POST JSON (adapter headers). false → failure('Failed to connect to AI service.'). Invalid JSON → failure('Invalid response from AI service: <json error>', ['raw'=>body]). Otherwise adapter->parseChatResponse (status code ignored). Exceptions go through safeExceptionMessage ("Internal server error" outside debug).
  • Anthropic adapter [VERIFIED: AnthropicAdapter.php:13-15, 23, 59-61]: headers x-api-key: <key>, anthropic-version: 2023-06-01, anthropic-beta: files-api-2025-04-14. Endpoint rtrim(base_url) . '/messages'. Payload {model: prompt.model ?? modelId, messages[], system?, max_tokens: options.max_tokens ?? options.max_completion_tokens ?? 4096, ...other options except response_format and max_completion_tokens}. Image blocks: data: URL → {type:image, source:{type:base64, media_type, data}}, else {source:{type:url}}; file blocks → document/file_id. Response: error.message → failure; content[0].text; usage mapped to prompt/completion/total; raw = the whole body (recognition reads raw.stop_reason).
  • OpenAI adapter [VERIFIED: OpenAIAdapter.php:13, 21, 50-62]: Authorization: Bearer <key>; /chat/completions; toApiPayload (system message first) minus response_format; max_tokens → max_completion_tokens; reasoning_effort: low for /^(o\d|gpt-5)/i unless set. Response: choices[0].message.content; raw.choices[0].finish_reason.
  • Content-Type: RequestSender adds Content-Type: application/json first; adapters append their headers. PHP json_encode escapes / and non-ASCII (ł) unless flagged. Upstream request-body parity: compare JSON semantically in the sidecar assertion, not byte-for-byte.
  • Settings [VERIFIED: models/Settings.php:13, 19-118]: $settingsCode = 'golem_settings' (not golem15_golem_settings as D-01 states). getModels() = array_filter(models, is_enabled), which preserves keys, so getDefaultModel()'s return $models[0] ?? null fallback returns null when model 0 is disabled even if others are enabled. Port the quirk. getVisionModel() = first enabled with accepts_images; also getImageModel (generates_images), getModelByName, getFileModel (has_files_endpoint). fields.yaml repeater fields: name, adapter (openai|anthropic, default openai), api_key (text, password attribute), base_url (default https://api.openai.com/v1), model, system_prompt, is_enabled, is_default, generates_images, accepts_images, has_files_endpoint, max_completion_tokens (default 4096). api_key is stored in plaintext: Settings has no encryption and the only encryption migration targets FaceSettings admin_password [VERIFIED: updates/version.yaml, encrypt_face_settings.php].
  • Recognition (AlbumRecognitionService.php): MAX_ALBUMS = 30, MAX_COMPLETION_TOKENS = 8192. Admin path with no config and no vision model → ApplicationException('No vision model configured in Golem AI settings.'). System prompt text (lines 237-292) with formats, genre hint (up to 20 distinct genres via albums in the collection, newest first) and language directive (en/pl from BCP-47). User message with the image. Strip fences, json_decode. On failure, one retry with the appended "IMPORTANT: Respond with raw JSON only…". A retry parse failure with finish reason length|max_tokens → RecognitionTruncatedException (controller → 200 {"albums":[],"code":"recognition_truncated"}), else []. Normalisation: drop nameless rows; nullableString fields; year 1889-2100; format must be in Album::FORMATS; tracklist via TracklistTextParser (Go tracklist_text_parser.go exists).
  • Not ported: FaceService, CompreFaceClient, ChatContextCollector, Conversations/Messages/Chat components (D-04). routes.php has no routes.
  • Settings storage gap (needs a decision, Open Question 2): cabana settings are typed singleton rows (id=1), scalar fields only. nestedValue is rejected, there is no repeater type in formFieldTypes, and unknown YAML keys fail boot [VERIFIED: cabana/settings.go:66-95, form_schema.go:22-44]. The project convention (Phase 5, user-resolved for fonoteka) is a dedicated typed table, not system_settings. Recommendation: table golem15_golem_models (one row per repeater item, sort_order, api_key lagoon.Encrypted json:"-"), edited through a cabana list+form controller under Settings › AI, plus an idempotent importer for the PHP system_settings row item='golem_settings' (a migration that runs only when system_settings exists, or a cutover command). This also reconciles D-03's "encrypted" with PHP's plaintext.

Feedback plugin surface (sm-feedback-plugin)

[VERIFIED: plugins/golem15/feedback/*, read this session]

  • Routes: group /_feedback/api/v1 with json.response: GET {key}/config (throttle:feedback-config = 60/min by IP), POST {key}/submit (throttle:feedback-submit = 10/min by IP), OPTIONS {any} → response('', 204) with where('any','.*'). A separate group with jwt.auth: PUT me/hidden. CORS paths include _feedback/api/* in both PHP and Go configs.
  • config: keyMatches = enabled && widget_key !== '' && hash_equals, else 404 {"error":true,"message":"Not found"}. Origin gate: no Origin header → allowed; unparseable host → false; empty allow-list → fail closed 403 {"error":true,"message":"Origin not allowed"}; host compared lowercase against getAllowedOriginHosts() (lines split on [\r\n]+, trimmed, parse_url host or the raw line, lowercased, unique). 200 {"success":true,"data":{"position","allowHide","colors":{"primary","accent"},"labels":{"title","placeholder","success"}}} with ?lang=en selecting _en, anything else _pl.
  • submit: validation (message required|string|max:5000, type required|in:bug,feature,other, email nullable|email|max:255, page_url required|string|max:2000, user_agent nullable|max:500, console_log nullable|max:20000, screenshot nullable|image|mimes:jpg,jpeg,png,gif,webp|max:10240) → 422 {"error":true,"message":"Validation failed","details":errors}. ImageContentGuard (finfo sniff in jpeg/png/gif/webp + getimagesize) → 422 with details.screenshot = ["The file is not a valid image."]. Create the submission, attach the screenshot (File, is_public = true, attachOne screenshot), dispatch SyncFeedbackToG15Office, 202 {"success":true}. Go classes/image_guard.go (IsAllowedImage, SniffImageMIME) can be copied into the plugin (PHP keeps per-plugin copies on purpose).
  • me/hidden: hidden required|boolean → 422 {"error":"Validation failed","errors":...}; UserPreference::setWidgetHidden (updateOrCreate) → 200 {"hidden":bool}.
  • getApiArray listener: feedback_widget_hidden => UserPreference::isWidgetHidden(id) (value of hidden, false when no row).
  • Settings: $settingsCode = 'golem15_feedback_settings' (line 14). initSettingsData: enabled=true, allow_hide=false, widget_key='wk_'.Str::random(32), position='bottom-right', g15office_task_priority='normal', and the pl/en label defaults. Rules are all nullable (position in:bottom-right,bottom-left,middle-right,middle-left, priority in:low,normal,high,urgent). Fields use colorpicker and readOnly, which cabana does not support (unknown keys fail boot). Adapt the Go fields.yaml (text fields, attributes: {readonly: …}) or add the types to cabana. Scalar-only, so a dedicated typed singleton table works with cabana settings. widget_key must be generated on first creation, because Winter's instance() persists defaults on first access [ASSUMED Winter behaviour].
  • Models and tables: golem15_feedback_submissions (id, message text, type string default 'bug', email, page_url 2000, user_agent, console_log longtext, status default 'pending' indexed, g15office_task_id, g15office_error text, timestamps). golem15_feedback_user_preferences (id, user_id unique, hidden bool default false, timestamps). Status values pending/sent/failed. Type map bug→Bug, feature→Feature, other→Task.
  • SyncFeedbackToG15Office: $tries = 3, $backoff = 30. Config feedback.g15_office.{base_url, token, project} from env G15_OFFICE_BASE_URL/TOKEN/PROJECT. createTask POST JSON {base}/_support/api/v1/projects/{project}/tasks with Bearer, Accept: application/json, body array_filter({title:"Feedback: "+Str::limit(collapsed message,80), description, type, status?, priority ?: 'normal'}). Then attachFile multipart POST {base}/_support/api/v1/tasks/{hashId}/attachments (field file). Decode: false → "connection error"; non-JSON → "invalid response body"; error truthy → message. Success: status sent, task id stored; failure: status failed, g15office_error, rethrow (retry). Description markdown format at lines 198-226. In Go: conga.MaxAttempts(3); River's backoff differs from 30 s (acceptable, not parity-visible) [ASSUMED].
  • embed.js (643 lines) is served at /plugins/golem15/feedback/assets/js/embed.js, which the Nuxt proxy uses (nuxt.config.ts:254, 369). No framework capability serves public plugin assets [VERIFIED: pact/capabilities.go interface list], so the plugin registers a GET route serving the embedded file.
  • No admin submissions list exists in PHP (no controller). D-13's list is a net-new cabana list controller, which is fine but not parity-checked.
  • Parity: feedback routes are outside fonoteka's routes.php (154 ids). Follow the userAPIRouteIDs/realtimeRouteIDs precedent in parity/check_corpus.go:619-630 and routes.snapshot (172 lines) by adding a feedbackRouteIDs set and an auth group.

Console commands

  • fonoteka:prune-notifications [VERIFIED: PruneNotifications.php:20-21]: Notification::where('created_at', '<', now()->subDays(90))->delete(); then $this->info("Pruned {$deleted} notifications older than 90 days.");, exit SUCCESS. Go table golem15_fonoteka_notifications. Register in Commands() (plugin.go:267). The scheduler entry already exists.
  • fonoteka:reindex {--drop-old-items-index} [VERIFIED: ReindexAlbums.php:16-17, 49-103]: not configured (scout.driver === typesense && Settings search_use_typesense && api_key) → error Typesense reindex skipped: enable Fonoteka Typesense search and configure TYPESENSE_API_KEY. FAILURE. Any album with collection_id IS NULL OR <= 0 → Album reindex aborted: every active Album must have a positive collection_id. FAILURE. Then scout:flush and scout:import (chunks of 500 [ASSUMED Scout default]). If the album table is empty, create the collection from the schema. Then search q=*, query_by=name, filter_by=collection_id:=0, per_page=1; found != 0 → Album reindex failed integrity check: collection_id:=0 documents exist. FAILURE. With --drop-old-items-index, delete golem15_fonoteka_items (Deleted legacy Typesense collection: … or, on ObjectNotFound, Legacy Typesense collection already absent: …). Success message Album index rebuilt; collection_id:=0 document count is zero. Go: beachcomber.From(app); Engine().Name(), Configured(), settingsGate{}.Enabled; Engine().Flush(index); batched Upsert(index, schema, docs) via album.ToSearchableArray; beachcomber.SearchPage for Found. Gap: Flush treats a missing index as nil, so the "already absent" message cannot be told apart. Add an optional beachcomber.IndexDropper (DropIndex(ctx, index) (existed bool, err)) to the framework (README + docs), or accept a single message (Open Question 6). "Before" per SRCH-02 is the DB check in PHP; "after" is the Typesense found check.
  • fonoteka:oauth-client shipped in Phase 8 (console/oauth_client.go).

Submodule workflow for the two new repos

  • Both remotes are reachable and empty: git ls-remote git@git.golem15.com:golem15/sm-golem-plugin.git and .../sm-feedback-plugin.git exit 0 with no refs, while a nonexistent repo errors with "Cannot find repository" [VERIFIED: probe this session]. Neither is mounted yet; .gitmodules lists only plugins/golem15/user [VERIFIED].
  • git submodule add of an empty remote cannot stage a gitlink (there is no commit to check out) [ASSUMED git behaviour]. Recommended sequence: git init the plugin directory at plugins/golem15/golem, write and commit the initial module there, git remote add origin …, push master (a user action in prior phases; executors were told not to push, see 13-VERIFICATION human item), then in fonoteka.go run git submodule add git@git.golem15.com:golem15/sm-golem-plugin.git plugins/golem15/golem (an existing repository at the path is staged without cloning). Commit .gitmodules and the gitlink separately.
  • Wiring (precedent: sm-user-plugin): plugin go.mod with module git.golem15.com/golem15/sm-golem-plugin, replace git.golem15.com/golem15/summercms => ../../../../summercms.go (feedback also replace git.golem15.com/golem15/sm-user-plugin => ../user). App go.work use ./plugins/golem15/golem and ./plugins/golem15/feedback. App go.mod require + replace … => ./plugins/golem15/<name>. The fonoteka plugin's go.mod requires sm-golem-plugin with replace => ../golem. summer.yaml adds ids golem15.golem and golem15.feedback (golem before fonoteka). Regenerate plugins.gen.go.
  • sm-user-plugin master is still 5 commits ahead of origin [VERIFIED: git status -sb], so a fresh clone cannot check out the pointer. That is unrelated to this phase but blocks CI clones.
  • README rules apply to plugin READMEs (never name the consuming application).

Architecture Patterns

System Architecture Diagram

Nuxt SPA / fonoteka-mcp / embed.js
        │ HTTPS
        ▼
surf router ── group middleware (jwt | inv_token+scope | public throttle) ──► controllers/api
        │                                                                     │
        │      in-controller limiter (surf.MemoryStore) ◄────────────────────┤
        │                                                                     ▼
        │                                        classes: gates/resolvers (DiscogsAllowed, ResolveAIConfig)
        │                                                  │                    │
        │                           ┌──────────────────────┘                    └──────────────┐
        │                           ▼                                                         ▼
        │                 Discogs client ──► rate limiter ──► Postgres UNLOGGED     sm-golem-plugin AIService
        │                     │                (atomic upsert)                          │ adapter (anthropic|openai)
        │                     ▼                                                         ▼
        │          fetchguard.Client (guarded) ──dial guard──► api.discogs.com    fetchguard.Client (guarded | trusted)
        │                     │                                                         │
        │                     └──► CoverImporter (AllowHosts *.discogs.com) ──► storage │──► AI provider
        ▼
DB transaction ── conga.Dispatch ──► river_job + summer_jobs
                                         │ LISTEN/NOTIFY
                                         ▼
                        River worker (serve or queue:work)
             ┌──────────────┬───────────────┬──────────────────┐
             ▼              ▼               ▼                  ▼
     CSV match job    CSV import job   digest job      feedback G15Office job
   (240 s; 429 →      (write service,  (delete row,    (fetchguard POST JSON +
    re-Dispatch       bulk broadcast)   postcard mail)   multipart, 3 attempts)
    with Delay)
fonoteka/plugins/golem15/fonoteka/
├── classes/discogs/      # client.go, rate_limiter.go, mapper.go, scorer.go, applicator.go,
│                         # import_resolver.go, input_parser.go, price_suggestion.go, cover_fetcher.go, errors.go
├── classes/recognition.go  # AlbumRecognitionService port (+ truncated error)
├── controllers/api/      # release_match_controller.go, wishlist_release_match_controller.go,
│                         # recognize_controller.go, discogs_import_controller.go, album_cover_fetch_controller.go
├── jobs.go (or jobs/)    # csv match/import, digest workers
├── console/              # prune_notifications.go, reindex.go
└── updates/              # discogs rate windows UNLOGGED table
plugins/golem15/golem/  (sm-golem-plugin)
├── plugin.go routes.go(no routes) classes/{service.go, providers/*.go, prompt.go, response.go,
│   prompt_factory.go, ssrf_guard.go} models/{ai_model.go} controllers/ (admin list/form) updates/ lang/en
plugins/golem15/feedback/  (sm-feedback-plugin)
├── plugin.go routes.go controllers/api/{feedback_api_controller.go, me_hidden_controller.go}
├── classes/{g15office_client.go, image_guard.go} jobs/sync_g15office.go models/ updates/ lang/{en,pl}
└── assets/js/embed.js (embedded, served by a route)

models/ stays a leaf package (the rule is enforced by tooling).

Pattern 1: Worker with summer_jobs outcome semantics

What: Translate PHP JobManager calls one-to-one: startJob → m.StartJob(ctx, id, total), updateJobState → UpdateJobState, completeJob(meta) → CompleteJob, failJob(meta) → FailJob then return nil (PHP does not retry), cancelJob inside a job → StopJob, checkIfCanceled → CheckIfCanceled. conga.JobID(ctx) gives the row id.

// Source: modules/conga/README.md Usage + fonoteka jobs.go pattern
conga.Job(p.matchCsv, conga.OnQueue(classes.CsvMatchQueue), conga.Timeout(240*time.Second))
// self re-dispatch on rate limit (inside the job):
next, err := m.Dispatch(ctx, gdb, classes.CsvMatchArgs{CsvImportID: imp.ID}, conga.DispatchOpts{
    Label: classes.CsvMatchLabel, Queue: classes.CsvMatchQueue, Count: pending,
    Metadata: map[string]any{"csv_import_id": imp.ID, "resumed_after_rate_limit": true, "retry_after": delay},
    Delay: time.Duration(delay) * time.Second,
})

Pattern 2: Injected clock (D-16)

type Clock interface {
    Now() time.Time
    Sleep(ctx context.Context, d time.Duration) error // returns ctx.Err() on cancel
}

The limiter's Acquire, the client's 429 loop and the job's delay computation take it. Tests use a fake whose Sleep advances Now. The Sleep respects ctx so the 240 s job timeout and request cancellation interrupt waits.

Pattern 3: Tier-aware AI config

classes.AIConfig gains a trust marker that only the AdminVisionModel adapter sets. Recognition and ai-credential/test call golem Send(ctx, prompt, cfg). The golem plugin selects fetchguard TrustedMode for admin configs and guarded PublicOnly for everything else, after AssertSafeURL(baseURL) (allowlist) when a user/org override is present.

Anti-Patterns to Avoid

  • Sleeping out the Discogs window inside the job: PHP explicitly never does this. Re-dispatch with Delay.
  • Making the Discogs base URI configurable: PHP keeps it a code literal for SSRF reasons. Inject the test transport instead.
  • {ok:false} for an unsafe base_url: PHP answers 500.
  • Marshalling multi-key job metadata via Go maps and comparing strings: see Pitfall 4.
  • Holding the CSV row lock across a Discogs call: see the WR-02 section.

Don't Hand-Roll

Problem Don't Build Use Instead Why
Private-IP / reserved-range checks New CIDR tables per consumer fetchguard dial Control (isReservedOrPrivate) Already a superset of PHP's lists, with NAT64/6to4 and zone handling
Inbound per-key counters Another bespoke map+mutex surf.MemoryStore.Attempt Atomic check+increment, returns retryAfter
Cover download allow-listing New downloader classes.CoverImporter / fetchguard AllowHostsMode Ported and parity-tested in Phase 12
Job progress/cancel bookkeeping Custom status table conga summer_jobs API job_contract.go and the parity row goldens already depend on it
Image sniffing New magic-byte parser classes/image_guard.go (copy into feedback) Matches PHP finfo + getimagesize parity cases
Cross-process rate window Advisory-lock dance One INSERT … ON CONFLICT DO UPDATE … WHERE … RETURNING Atomic per Postgres docs
Tracklist parsing in recognition New parser tracklist_text_parser.go PHP reuses one parser (D-22)

Key insight: almost every primitive exists. Phase risk lies in faithful edge-case parity (ordering of checks, error-to-status maps, metadata shapes) and in the test harness for upstream calls, not in new infrastructure.

Common Pitfalls

Pitfall 1: Allowlist omitted from the user/org base_url guard

What goes wrong: Go accepts an https://api.groq.com override that PHP rejects (500); parity diff and SSRF surface widen. Avoid: port SSRFGuard fully (scheme, allowlist, IP) before the dial guard. Warning sign: no allowed_hosts config key in sm-golem-plugin.

Pitfall 2: PHP 500 vs {ok:false} on guard failure

What goes wrong: Go's "nicer" error handling returns 200 {ok:false}. Avoid: record cases 16 (unsafe and non-allowlisted base_url) and assert the Winter 500 page.

Pitfall 3: Settings key and plaintext key mismatch

What goes wrong: Importing from system_settings with item golem15_golem_settings finds nothing; PHP's is golem_settings. Or the importer expects ciphertext. Avoid: Open Question 2 resolved before planning the storage.

Pitfall 4: Job metadata key order

What goes wrong: conga Metadata map[string]any marshals keys sorted, while PHP preserves insertion order ({"paused","retry_after","next_job_id"}). nuxt-csv.rows.json compares the metadata string [VERIFIED: parity/fonoteka_flows_test.go:84-95]. Avoid: compare metadata JSON semantically in new goldens, or extend conga to accept an ordered value. Single-key metadata is unaffected.

Pitfall 5: Unregistered-kind assertions break when workers land

What goes wrong: TestJobContractDispatchWhileWorkerRuns/TestCsvJobRows and conga's ErrUnregisteredKindQueue assumptions (queues "unserved") flip once the three kinds register. Avoid: update the tests in the same plan. Check that nuxt-csv still expects cancelled River states (the parity app handler does not start a worker).

Pitfall 6: Retrying jobs PHP never retries

What goes wrong: returning an error after FailJob makes River retry, which double-processes rows. Avoid: PHP catches Throwable and returns; Go returns nil after FailJob. Only the 240 s timeout or a panic should surface as a River retry.

Pitfall 7: Discogs lock-held fetch in row edit

See the WR-02 section: fetch outside the lock, then re-check under the lock.

Pitfall 8: getDefaultModel key-preservation quirk

array_filter keeps keys, so $models[0] ?? null can be null while enabled models exist. Port the quirk deliberately and test it.

Pitfall 9: Recognize manifest status mismatch

The manifest case says 422 but the fixture shows 403 for POST albums/recognize jwt. TestCheckCorpusPortedCaseStatus will fail on flip unless fixed.

Pitfall 10: Upstream JSON byte equality

PHP json_encode escapes / and Unicode. Go escapes differently (json/v2 in Go 1.27). Assert upstream bodies semantically (normalise, then compare). Response bodies to our clients stay byte-compared via the existing wire helpers.

Pitfall 11: cabana rejects unknown YAML keys

The feedback colorpicker/readOnly and the golem repeater fail boot. Adapt the YAML or add the types to cabana (a framework change with docs).

Code Examples

Redacting handler skeleton

// Source: log/slog Handler contract (stdlib); key list from RedactCredentialsTap.php
type redactHandler struct{ next slog.Handler }
func (h redactHandler) Enabled(ctx context.Context, l slog.Level) bool { return h.next.Enabled(ctx, l) }
func (h redactHandler) Handle(ctx context.Context, r slog.Record) error {
    out := slog.NewRecord(r.Time, r.Level, scrub(r.Message), r.PC)
    r.Attrs(func(a slog.Attr) bool { out.AddAttrs(redactAttr(a)); return true })
    return h.next.Handle(ctx, out)
}
func (h redactHandler) WithAttrs(as []slog.Attr) slog.Handler { /* redact each, then h.next.WithAttrs */ }
func (h redactHandler) WithGroup(n string) slog.Handler     { return redactHandler{h.next.WithGroup(n)} }

Inbound limiter (Laravel tooManyAttempts + hit)

// Source: modules/surf/limiter_store.go Store.Attempt
ok, _, retry := store.Attempt("fonoteka-discogs-missing:"+strconv.FormatUint(uint64(user.ID), 10), 60, time.Minute)
if !ok {
    return wire.JSON(w, 429, ordered("result","error","code","too_many_requests","retry_after", int(math.Ceil(retry.Seconds()))))
}

(wire.JSON/ordered stand for the app's existing ordered-JSON helpers; use whatever the Phase 12/13 controllers use.)

State of the Art

Old Approach Current Approach When Changed Impact
Apparatus RequestSender (curl, resolve-time check, follows redirects, no POST guard) fetchguard client with dial-time guard This phase Closes DNS-rebind and redirect SSRF gaps
Winter SettingsModel over system_settings Dedicated typed tables + cabana settings Phase 5 decision Golem repeater and feedback settings need storage choices
Laravel sync/redis queue River via conga (LISTEN/NOTIFY) Phase 11 Delayed re-dispatch is native (Delay)

Deprecated/outdated: REQUIREMENTS.md INTG-02 "Anthropic Go SDK" (reword per D-06); API-08 "sitemap" (reword per D-13/D-14); ROADMAP SC4/SC5/SC6 (reword per D-06/D-07/D-14).

Assumptions Log

# Claim Section Risk if Wrong
A1 Guzzle honours HTTPS_PROXY in non-CLI SAPIs and libcurl honours https_proxy for raw curl, so a recording proxy captures PHP's Discogs and AI calls D-11 Recording proxy needs another capture route (Laravel Http events for Discogs; admin-tier local base_url for AI)
A2 git submodule add cannot stage an empty remote; pre-initialising the directory works Submodules Minor workflow change
A3 Winter SettingsModel::instance() persists initSettingsData() defaults on first access (widget_key stable) Feedback widget_key generation timing differs
A4 AI request timeout 120 s, G15Office 30 s fetchguard Hung upstream holds a request or worker longer or shorter
A5 Scout imports in chunks of 500 Reindex Batch size only
A6 River backoff instead of a fixed 30 s for the G15Office job is acceptable Feedback Not parity-visible
A7 The UNLOGGED table schema and SQL shown for the limiter D-17 Design detail to settle in the plan
A8 No FK or trigger links summer_jobs/river_job back to csv_imports (no deadlock under the row lock) WR-02 Lock ordering issue
A9 The fetchguard client API shape (NewClient, modes, helpers, transport seam) fetchguard Plan-level design

Open Questions (RESOLVED)

  1. Is GOLEM15_SSRF_ALLOWED_HOSTS set in production? We know the default allowlist rejects any user/org base_url outside *.openai.com (Anthropic included). The .env is secret-guarded. Recommendation: ask the user for the production value (names only) and make it the Go config default for the application; keep the PHP default in the plugin. DEFERRED (Phase 15 cutover): 14-04 keeps the PHP default allowlist and honours the env override. The operator confirms the production value at cutover.
  2. Golem settings storage and key. We know the PHP code is golem_settings with a plaintext api_key inside a repeater JSON, and cabana has no repeater. D-01 says golem15_golem_settings; D-03 says "encrypted" and "reading the existing settings row". Recommendation: confirm a dedicated golem15_golem_models table (encrypted key, admin list/form) plus an importer from system_settings item golem_settings; correct D-01's key string. RESOLVED: D-18 (user, 2026-10-03): the golem15_golem_models table with an encrypted key, plus an importer from golem_settings.
  3. Feedback settings storage. The same convention question for golem15_feedback_settings (scalar, so a typed singleton table fits cabana). Confirm a typed table plus an importer. RESOLVED: D-18: a typed singleton table plus an importer (14-05).
  4. Recording-proxy scope (D-15). Building summer parity:upstream (MITM CA, script/forward modes) is the only faithful way to capture PHP's actual upstream requests. Confirm it is in scope, or accept hand-authored sidecars. RESOLVED: D-19 (user): summer parity:upstream is in scope (14-01).
  5. Phase 10.1 admin Discogs stubs (discogsLookup, discogsSync) say "Phase 14 replaces". PHP has no such admin actions. In or out of scope? RESOLVED: out of scope. The stubs are left unchanged and 14-03 records this as an assumption.
  6. Reindex "already absent" message: add beachcomber.IndexDropper (framework change) or accept one message for both outcomes? RESOLVED: beachcomber.IndexDropper and EnsureIndex are added in 14-01.
  7. Roadmap home for D-09 routes (oauth-identities, /api/v1/fonoteka/me): Phase 15 requires all routes green; a todo exists, and a phase needs to be named. DEFERRED: tracked in .planning/todos/pending/orphan-pending-routes.md. A phase must be inserted before Phase 15, for example with /gsd-phase --insert 14.1. The 14-06 gate pins exactly these 3 routes as pending.

Environment Availability

Dependency Required By Available Version Fallback
Go everything ✓ go1.27.0 —
Docker daemon testcontainers (Postgres tests) ✓ 29.7.2 -short skips
PHP CLI recording PHP parity cases ✓ 8.5.10 —
sqlite3 php_parity.sh rows goldens ✓ 3.53.4 —
node capture_clients.mjs ✓ v22.23.2 —
psql manual DB checks ✓ 18.6 —
ssu submodule management ✓ /home/jin/.local/bin/ssu git submodule
git.golem15.com sm-golem-plugin / sm-feedback-plugin D-01 / D-12 ✓ reachable, empty — —
Live Discogs / Anthropic / OpenAI / G15Office one-time forward-mode recording only not probed — script-mode sidecars (no live calls)
fonoteka.go builds baseline ✓ go build ./... ok — —

Missing dependencies with no fallback: none. Note: pushing the new plugin repos and sm-user-plugin (5 ahead) is a user action.

Validation Architecture

Test Framework

Property Value
Framework Go testing (+ testify assertions in existing tests), testcontainers-go v0.44.0 postgres, tide replay
Config file none (Go); gate script summercms.go/scripts/check-phase14.sh (new, modelled on check-phase13.sh: --self-test --go --parity --named --coverage --evidence --all, EXPECTED_PORTED/EXPECTED_PENDING, coverage floor 80, -race on app packages)
Quick run command go -C ../fonoteka.go test ./plugins/golem15/fonoteka/... -count=1 -short
Full suite command go test ./... -count=1 && go -C ../fonoteka.go test ./... -count=1 -race && bash scripts/check-phase14.sh --all

Prior verify-command pattern (Phase 13 plans): go -C ../fonoteka.go vet ./... && go -C ../fonoteka.go test ./plugins/golem15/fonoteka -run '^(TestX|TestY)$' -count=1 -race -v && go -C ../fonoteka.go test ./parity -run '^(TestParityCorpus|TestFonotekaNuxtFlows)$' -count=1 -v && go -C ../fonoteka.go run ./parity/check_corpus.go --manifest parity/manifest.yaml --routes /media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/routes.php --require-recorded --check-secrets. Framework tasks add go test ./cmd/summer -count=1 -run '^(TestDocsTree|TestDocsCommandsMirrorGeneratedMain)$' && go run ./cmd/summer docs:build --check.

Phase Requirements → Test Map

Req ID Behavior Test Type Automated Command File Exists?
JOBS-02 match job: 0/1/many results, gate off, duplicate, cancel, superseded, 429 → re-dispatch with delay = max(retryAfter, secondsUntilAvailable), timeout 240 s as a value unit + Postgres go -C ../fonoteka.go test ./plugins/golem15/fonoteka -run '^TestCsvMatchJob' -count=1 -race ❌ Wave 0
JOBS-02 import job: canonical overwrite/fill/create, draft vs CSV merge, write_failed row, unauthorized → canceled, bulk_updated publish once unit + Postgres ... -run '^TestCsvImportJob' ❌
JOBS-02 WR-02: commit vs mapping interleave → at most one import job Postgres ... -run '^TestCsvWR02' ❌
JOBS-03 digest: skipped row, sent count, row deleted, en/pl template, wishlist kind guard unit + Postgres ... -run '^TestWishlistDigestJob' ❌
SRCH-02 reindex: not configured, zero-tenant DB abort, found≠0 failure, drop legacy present/absent unit (fake engine) ... -run '^TestReindexCommand' (fake engine exists: fake_engine_test.go) ❌
CLI-05 prune deletes >90 d, message text, schedule entry resolves unit + Postgres `... -run '^(TestPruneNotifications TestSchedule)'`
INTG-01 limiter: threshold, budget exhaustion carries window remainder, tighten-only header sync, retry-after fallback, fake clock unit + Postgres SQL go -C ../fonoteka.go test ./plugins/golem15/fonoteka/classes/discogs -count=1 -race ❌
INTG-01 mapper/parser/scorer/price resolver vs PHP truth tables unit ... ./classes/discogs -run '^TestPHPTruth' ❌ (generator parity/discogs_truth_tables.php)
INTG-01 routes parity with sidecars (cases 1-15) parity `go -C ../fonoteka.go test ./parity -run '^(TestParityCorpus TestCheckCorpusPortedCaseStatus)$' -count=1`
INTG-02 adapters build PHP payloads (headers, max_tokens mapping, reasoning_effort, image blocks), parse errors verbatim unit go -C ../fonoteka.go test ./plugins/golem15/golem/... -count=1 ❌
INTG-02 SSRF guard: scheme, allowlist suffix, private IP; admin trusted bypass; 500 page on guard failure unit + parity ... -run '^TestSSRFGuard' + parity case 16 ❌
INTG-02 recognition: retry, truncation code, normalisation caps, admin tier unit + parity go -C ../fonoteka.go test ./plugins/golem15/fonoteka -run '^TestRecognize' ❌
API-08 feedback config/submit/options/me-hidden, origin gate, image guard, getApiArray key, G15Office job via fake unit + parity go -C ../fonoteka.go test ./plugins/golem15/feedback/... -count=1 -race ❌
framework fetchguard client modes, redirects, caps, transport seam; redact handler; tide sidecar replay asserts unit go test ./modules/fetchguard/... ./modules/tide/... -count=1 ❌

Sampling Rate

  • Per task commit: the task's named -run tests plus go vet in the touched repo.
  • Per wave merge: go test ./... in both repos plus TestParityCorpus/TestFonotekaNuxtFlows.
  • Phase gate: scripts/check-phase14.sh --all green before /gsd-verify-work.

Wave 0 Gaps

  • fonoteka.go/parity/discogs_truth_tables.php (truth-table generator, csv_truth_tables.php pattern)
  • tide sidecar loader and fake; the replayPortedRoute hook for *.upstream.yaml
  • Fake clock helper shared by limiter, client and job tests
  • Test harness to run a conga worker function directly (call the job func with a ctx carrying JobID) without a live River client
  • scripts/check-phase14.sh (copy the check-phase13 structure; update EXPECTED_PORTED/EXPECTED_PENDING: today 157/14, Phase 14 flips 11 pending routes (14 minus the 3 D-09 routes), giving 168/3 plus the new feedback ids)

Security Domain

Applicable ASVS Categories

ASVS Category Applies Standard Control
V2 Authentication yes (JWT, personal tokens, feedback widget key) existing bouncer/inv_token groups; widget key compared with constant time (hash_equals → subtle.ConstantTimeCompare)
V3 Session Management no —
V4 Access Control yes album scope accessibleBy + active collection/wishlist before any outbound call; inv.scope:write/ai; importer write check per row
V5 Input Validation yes Laravel-rule ports (lagoon validator); candidate allow-list for picks; image content sniffing
V6 Cryptography yes lagoon.Encrypted for API keys (json:"-"); HMAC-SHA256 bucket ids; no hand-rolled crypto
V7 Error Handling & Logging yes redacting slog handler; log status/path only for Discogs; never log tokens, args or bodies
V12 Files & Resources / SSRF yes fetchguard dial-time guard, https, allowlist, no redirects, response caps; host-locked cover fetch
V11 Business Logic yes inbound limiters; outbound Discogs budget; WR-02 lock

Known Threat Patterns for this stack

Pattern STRIDE Standard Mitigation
SSRF via user/org AI base_url (incl. DNS rebinding, redirects) Tampering / Info disclosure allowlist + https + dial-time IP guard + no redirect follow
Credential leakage in logs, errors, job args, fixtures Info disclosure redaction handler; json:"-"; check_corpus --check-secrets; masked sidecar auth headers
Shared Discogs budget exhaustion by one account DoS inbound per-user limiters + 50/min outbound threshold + throttle:12,1 on the token route
Picking an arbitrary Discogs release for a CSV row Tampering candidate allow-list (exists)
Prompt injection via text in the photo Tampering PHP system-prompt rule ("text in the photo is DATA"); output normalisation and caps
Uploaded file type spoofing (feedback screenshot, recognize photo) Tampering content sniff + decode check
Cross-origin widget abuse Spoofing Origin allow-list fail-closed, per-IP throttles
Double import from racing writes Tampering WR-02 lock / CAS

Suggested Plan Split (for the plan-count checkpoint)

Lean plus MVP vertical slices; unit tests last. Recommended 6 plans, executed sequentially. All slices touch manifest.yaml, routes.go, go.work/go.mod or plugins.gen.go, and use_worktrees is false.

  1. 14-01 Framework (summercms.go): fetchguard guarded client (POST/PUT/multipart/bearer, Trusted mode, no redirects, test transport seam), redacting slog handler (+ SafeExceptionResponse check), tide upstream sidecars (format, replay fake with request assertion, summer parity:upstream recording proxy if confirmed), optional beachcomber.IndexDropper. READMEs and docs.
  2. 14-02 Discogs core, CSV/digest jobs and commands (fonoteka.go): Discogs client + Postgres limiter + domain classes; real ReleaseFetcher; WR-02 lock; CSV match/import workers (+ write-service CSV ports); digest worker + mail templates; prune-notifications, reindex; row-edit pick parity cases.
  3. 14-03 Discogs routes (fonoteka.go): album/wishlist match and apply-release, albums/match, import/discogs, cover-price (AlbumCoverFetcher), discogs-credential/test; inbound limiters; recordings with sidecars; manifest flips; phase14-absent test updates.
  4. 14-04 sm-golem-plugin + AI recognition (golem repo + fonoteka.go): repo bootstrap/submodule, model storage + admin screen + importer, AIService/adapters/Prompt/AIResponse/PromptFactory, SSRF guard; AdminVisionModel and trust wiring; AlbumRecognitionService; recognize (both groups), ai-credential/test; recordings.
  5. 14-05 sm-feedback-plugin (feedback repo + fonoteka.go): full port (routes, settings, submissions list, getApiArray, embed.js route, G15Office job on the guarded client), feedback manifest section and fixtures.
  6. 14-06 Unit tests and gate: full coverage for all Phase 14 code, PHP truth tables, check-phase14.sh, requirement and roadmap rewording checks, validation evidence.

Alternative (7 plans): split 14-02 into "Discogs core + CSV jobs" and "digest + commands" if 14-02 proves too large at planning time.

Sources

Primary (HIGH confidence, read this session)

  • PHP: fonoteka/plugins/golem15/fonoteka/{routes.php, jobs/*, console/*, classes/discogs/*, classes/{AlbumRecognitionService,AiConfigResolver,AiGate,UserAiConfig,OrgAiConfig,AiDefaults,NotificationService}.php, controllers/api/{AlbumReleaseMatch,WishlistReleaseMatch,RecognizeApi,DiscogsImport,AlbumCoverFetch,AiCredential,DiscogsCredential,CsvImportApi}Controller.php, config/fonoteka.php}
  • PHP: golem/{Plugin.php, models/Settings.php, models/settings/fields.yaml, classes/services/AIService.php, classes/providers/*, classes/valueobjects/{Prompt,AIResponse}.php, classes/security/SSRFGuard.php, config/ssrf.php, updates/*}
  • PHP: feedback/* (all files), apparatus/classes/{RequestSender.php, traits/SafeExceptionResponse.php, logging/RedactCredentialsTap.php}
  • Nuxt vue-fonoteka-app/{nuxt.config.ts, app/stores/fonoteka.ts, app/stores/auth.ts}, MCP fonoteka-mcp/src/client.ts
  • Go: fonoteka.go (job_contract.go, jobs.go, csv_import_service.go, wishlist_notifications.go, ai_config_resolver.go, gates.go, cover_importer.go, search.go, schedule.go, console/oauth_client.go, routes.go, parity/*), summercms.go/modules/{fetchguard,conga,beachcomber,cabana,surf,lagoon/attach,pact}
  • Planning: 13-REVIEW.md (WR-02), 13-VERIFICATION.md, 12/13-CONTEXT.md, notes (core-plugins-own-repos, plugin-layout, apparatus-dissolved), todos
  • postgresql.org/docs/current/sql-insert.html: ON CONFLICT atomicity and RETURNING of non-updated rows
  • pkg.go.dev/net/http#Client: redirect semantics, GetBody, ErrUseLastResponse

Secondary (MEDIUM)

  • Remote reachability probe (git ls-remote) of both new repos

Tertiary (LOW)

  • Discogs rate limits (search summary of discogs.com/developers; direct fetch returned 403)

Metadata

Confidence breakdown:

  • Standard stack: HIGH (no new dependencies; every module read)
  • Architecture: MEDIUM-HIGH (seams verified; new framework API shapes are proposals)
  • Pitfalls: HIGH (each grounded in a file read this session)

Research date: 2026-10-03 Valid until: 2026-11-02 (stable in-repo facts); re-check vendor API details at implementation.