docs(14): add pattern map
This commit is contained in:
Jakub Zych
@@ -0,0 +1,470 @@
|
|||||||
|
# Phase 14: Domain jobs and external integrations - Pattern Map
|
||||||
|
|
||||||
|
**Mapped:** 2026-10-03
|
||||||
|
**Files analyzed:** ~45 new/modified (grouped by the 6 confirmed plans)
|
||||||
|
**Analogs found:** 40 / 45
|
||||||
|
|
||||||
|
Path roots used below:
|
||||||
|
- `FW` = `/media/nvme/dev/golem15/summercms.io/summercms/summercms.go`
|
||||||
|
- `APP` = `/media/nvme/dev/golem15/summercms.io/summercms/fonoteka.go`
|
||||||
|
- `FON` = `APP/plugins/golem15/fonoteka`
|
||||||
|
- `USR` = `APP/plugins/golem15/user` (sm-user-plugin submodule; analog for both new submodule plugins)
|
||||||
|
|
||||||
|
All analogs below are git-tracked (checked `git ls-files` in fonoteka.go and inside the user submodule).
|
||||||
|
|
||||||
|
## File Classification
|
||||||
|
|
||||||
|
| New/Modified File | Role | Data Flow | Closest Analog | Match |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| **Plan 14-01 (framework)** |||||
|
||||||
|
| `FW/modules/fetchguard/client.go` (new: NewClient, Do, PostJSON/PutJSON/PostMultipart, Bearer, TrustedMode, test transport seam) | utility | request-response | `FW/modules/fetchguard/fetch.go` | exact (same package) |
|
||||||
|
| `FW/modules/fetchguard/policy.go` (add `TrustedMode`, ClientPolicy) | config | — | itself | modify |
|
||||||
|
| `FW/modules/fetchguard/README.md`, `FW/docs/services/outbound-http.md`, `docs/architecture/introduction.md`, `docs/setup/coming-from-wintercms.md` | docs | — | existing README | modify |
|
||||||
|
| redacting slog handler (new small module, e.g. `FW/modules/<name>/redact.go` + README + root README row) | middleware (log) | transform | none in repo (stdlib slog contract) | no analog — use RESEARCH Code Example |
|
||||||
|
| `FW/modules/tide/upstream.go` (sidecar `*.upstream.yaml` load + replay fake asserting requests) | test utility | request-response | `FW/modules/tide/centrifugo.go` (fake HTTP recorder) | role-match |
|
||||||
|
| `FW/modules/tide/upstream_proxy.go` (CONNECT/MITM recording proxy) | utility | streaming/proxy | `FW/modules/tide/proxy.go` | role-match |
|
||||||
|
| `FW/cmd/summer/parity.go` (add `parity:upstream` command) | CLI | request-response | `parityBroadcastsCommand` in same file | exact |
|
||||||
|
| `FW/modules/beachcomber/searchable.go` (optional `IndexDropper`) + `typesense/engine.go` `DropIndex` | interface/engine | CRUD | `PageSearcher` + `SearchPage` in `searchable.go:87-108`; `Engine.Flush` `typesense/engine.go:205-215` | exact |
|
||||||
|
| **Plan 14-02 (Discogs core, jobs, commands)** |||||
|
||||||
|
| `FON/classes/discogs/client.go`, `rate_limiter.go`, `errors.go` | service | request-response | `FON/classes/cover_importer.go` (config + fetchguard + injectable fetch) | role-match |
|
||||||
|
| `FON/classes/discogs/{mapper,scorer,input_parser,price_suggestion,import_resolver,applicator,cover_fetcher}.go` | utility (pure transform) | transform | `FON/classes/csv/*` (PHP truth-table ported pure classes) | role-match |
|
||||||
|
| `FON/updates/<ts>_discogs_rate_windows.go` (UNLOGGED table) | migration | — | `FON/updates/11_secrets_slice.go` | exact |
|
||||||
|
| `FON/classes/csv_import_service.go` (WR-02 locks; real ReleaseFetcher install) | service | CRUD | `CommitCsvImport` same file lines 825-882 | exact |
|
||||||
|
| `FON/classes/album_write_service.go` (CSV variants) | service | CRUD | same file `CreateAlbum`/`UpdateAlbum` | exact |
|
||||||
|
| `FON/jobs.go` (3 new workers) | job | event-driven | `FON/jobs.go` existing mail workers | exact |
|
||||||
|
| `FON/mail.go` + `FON/views/mail/wishlist_subscription_digest(-en).htm` | config/template | — | existing `mailWishlistItemPurchased` registration in `mail.go` | exact |
|
||||||
|
| `FON/console/prune_notifications.go`, `FON/console/reindex.go` | CLI | batch | `FON/console/oauth_client.go` | exact |
|
||||||
|
| `FON/plugin.go` `Commands()` / Boot wiring | config | — | `plugin.go:267-272` | exact |
|
||||||
|
| `FON/schedule.go` (comment only) | config | — | itself | modify |
|
||||||
|
| `FON/config/config.yaml` (discogs.user_agent, rate_threshold, wait_budget_seconds, retry_after_fallback_seconds) | config | — | existing `discogs.*` keys lines 22-36 | exact |
|
||||||
|
| `APP/parity/discogs_truth_tables.php` | test generator | batch | `APP/parity/csv_truth_tables.php` | exact |
|
||||||
|
| **Plan 14-03 (Discogs routes)** |||||
|
||||||
|
| `FON/controllers/api/{release_match,wishlist_release_match,discogs_import,album_cover_fetch}_controller.go`, discogs-credential test action in `credentials_controller.go` | controller | request-response | `FON/controllers/api/credentials_controller.go` | exact |
|
||||||
|
| `FON/routes.go` | route | — | `routes.go:194-204` (credential routes), `:240` token group | exact |
|
||||||
|
| `FON/routes_table_phase13_test.go` (`phase14Absent` shrinks) | test | — | itself lines 270-282 | modify |
|
||||||
|
| `APP/parity/manifest.yaml`, `APP/parity/fixtures/routes/*.yaml` + `*.upstream.yaml` | fixture | — | existing route fixtures | exact |
|
||||||
|
| **Plan 14-04 (sm-golem-plugin + recognition)** |||||
|
||||||
|
| `APP/plugins/golem15/golem/{go.mod,plugin.go,README.md}` | plugin root | — | `USR/go.mod`, `USR/plugin.go` | exact |
|
||||||
|
| `golem/models/ai_model.go` (`golem15_golem_models`, encrypted api_key) | model | CRUD | `FON/models/user_ai_credential.go` | exact |
|
||||||
|
| `golem/updates/*` (table + importer from `system_settings` item `golem_settings`) | migration | batch | `FON/updates/11_secrets_slice.go`, `FON/updates/registry.go` | exact |
|
||||||
|
| `golem/controllers/*` + `controllers/models/config_list.yaml`/`config_form.yaml` + `models/ai_model/{fields,columns}.yaml` | admin controller | CRUD | `FON/controllers/genres_admin_controller.go` + `controllers/genres/config_list.yaml`; `FON/admin.go` (AdminFS embed) | exact |
|
||||||
|
| `golem/classes/{service.go,prompt.go,response.go,prompt_factory.go}` | service | request-response | `FON/classes/ai_config_resolver.go` (AIConfig shape) | partial |
|
||||||
|
| `golem/classes/providers/{anthropic,openai}.go` | adapter | request-response | none (new hand-rolled JSON over fetchguard client) | no analog |
|
||||||
|
| `golem/classes/ssrf_guard.go` (`AssertSafeURL`, allowlist) | utility | validation | `FON/classes/cover_importer.go` `allowedURL` + `fetchguard.hostAllowed` | role-match |
|
||||||
|
| `FON/classes/ai_config_resolver.go` (add `Trusted bool json:"-"`; AdminVisionModel set from golem Boot) | seam | — | itself lines 30-45; `SetReleaseFetcher` pattern | modify |
|
||||||
|
| `FON/classes/recognition.go` + `FON/controllers/api/recognize_controller.go`, ai-credential test action | service + controller | request-response | `credentials_controller.go`; `tracklist_text_parser.go` | role-match |
|
||||||
|
| `APP/go.work`, `APP/go.mod`, `APP/summer.yaml`, `APP/plugins.gen.go`, `APP/.gitmodules`, `FON/go.mod` | config | — | existing user-plugin entries | exact |
|
||||||
|
| **Plan 14-05 (sm-feedback-plugin)** |||||
|
||||||
|
| `APP/plugins/golem15/feedback/{go.mod,plugin.go,routes.go,README.md}` | plugin root/route | — | `USR/plugin.go`, `USR/routes.go` | exact |
|
||||||
|
| `feedback/controllers/api/{feedback_api_controller,me_hidden_controller}.go` | controller | request-response, file-I/O (multipart) | `FON/controllers/api/credentials_controller.go`; `album_photos_controller.go` (multipart upload) | role-match |
|
||||||
|
| `feedback/models/{submission,user_preference,settings}.go` | model | CRUD | `FON/models/settings.go`, `FON/models/user_ai_credential.go` | exact |
|
||||||
|
| `feedback/updates/*` (tables + settings importer) | migration | — | `FON/updates/11_secrets_slice.go` | exact |
|
||||||
|
| `feedback/admin_settings.go` + `models/settings/fields.yaml` | config | — | `FON/admin_settings.go` | exact |
|
||||||
|
| submissions admin list | admin controller | CRUD | `FON/controllers/genres_admin_controller.go` | exact |
|
||||||
|
| `feedback/classes/image_guard.go` | utility | validation | `FON/classes/image_guard.go` (copy) | exact |
|
||||||
|
| `feedback/classes/g15office_client.go` + `feedback/jobs/sync_g15office.go` | service + job | request-response / event-driven | `FON/jobs.go` (`conga.Job` + `MaxAttempts`) | role-match |
|
||||||
|
| GetApiArray listener in feedback `Boot` | event listener | pub-sub | `FON/plugin.go:90-103` | exact |
|
||||||
|
| embed.js route (`go:embed assets/js/embed.js`) | route | file-I/O | `USR/plugin.go` `//go:embed` blocks + `FON/admin.go` embed | role-match |
|
||||||
|
| `APP/parity/check_corpus.go` (`feedbackRouteIDs`), `routes.snapshot` | test tooling | — | `userAPIRouteIDs`/`realtimeRouteIDs` lines 21-45, 619-630 | exact |
|
||||||
|
| **Plan 14-06 (tests + gate)** |||||
|
||||||
|
| `FW/scripts/check-phase14.sh` | gate script | batch | `FW/scripts/check-phase13.sh` | exact |
|
||||||
|
| worker tests | test | event-driven | `FON/job_contract_worker_test.go` | exact |
|
||||||
|
| inbound limiter / clock tests | test | — | `FON/classes/public_share.go` `PubfailCounter` (injected `now`) | role-match |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan 14-01 — Framework (summercms.go)
|
||||||
|
|
||||||
|
### `modules/fetchguard/client.go` (utility, request-response)
|
||||||
|
**Analog:** `FW/modules/fetchguard/fetch.go` (178 lines) + `policy.go`.
|
||||||
|
|
||||||
|
Reuse verbatim: URL/scheme validation (lines 41-51), `resolveLimits` (lines 103-132), `dialControl` (lines 148-169), `mapTransportError` (171-178), and the transport construction:
|
||||||
|
```go
|
||||||
|
// fetch.go:56-75
|
||||||
|
client := &http.Client{
|
||||||
|
Timeout: timeout,
|
||||||
|
CheckRedirect: func(*http.Request, []*http.Request) error {
|
||||||
|
return http.ErrUseLastResponse
|
||||||
|
},
|
||||||
|
Transport: &http.Transport{
|
||||||
|
// User-supplied URLs must not be forwarded through HTTP_PROXY:
|
||||||
|
Proxy: nil,
|
||||||
|
DialContext: (&net.Dialer{
|
||||||
|
Timeout: timeout,
|
||||||
|
Control: dialControl(policy),
|
||||||
|
}).DialContext,
|
||||||
|
TLSClientConfig: policy.tlsConfig,
|
||||||
|
DisableKeepAlives: true,
|
||||||
|
ForceAttemptHTTP2: true,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Body cap pattern (fetch.go:85-96): `io.ReadAll(io.LimitReader(resp.Body, maxBytes+1))` then `ReasonTooLarge`. Return status, do not judge it (`Result{Body, ContentType, StatusCode}`).
|
||||||
|
|
||||||
|
Mode enum to extend (policy.go:12-18):
|
||||||
|
```go
|
||||||
|
type Mode int
|
||||||
|
const (
|
||||||
|
AllowHostsMode Mode = iota
|
||||||
|
PublicOnlyMode
|
||||||
|
)
|
||||||
|
```
|
||||||
|
Add `TrustedMode` after `PublicOnlyMode` (keeps existing numeric values). Test hooks today are unexported fields (`tlsConfig`, `skipReservedCheck`, policy.go:28-35); the new transport seam must also be unreachable from production input (code-only option). Error type `*Error{Reason, Err}` (policy.go:38-58) reused unchanged. Tests: copy `withTestLoopback` usage from `fetch_test.go`.
|
||||||
|
|
||||||
|
### `cmd/summer/parity.go` — `parity:upstream`
|
||||||
|
**Analog:** `parityBroadcastsCommand` (parity.go:70-90):
|
||||||
|
```go
|
||||||
|
func parityBroadcastsCommand() bonfire.Command {
|
||||||
|
return bonfire.Command{
|
||||||
|
Name: "parity:broadcasts",
|
||||||
|
Description: "Record the Centrifugo publications ...",
|
||||||
|
Flags: []bonfire.Flag{
|
||||||
|
{Name: "listen", Description: "Loopback address of the fake Centrifugo recorder", Default: tide.DefaultCentrifugoListen},
|
||||||
|
...
|
||||||
|
},
|
||||||
|
Run: runParityBroadcasts,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Use `requireFlag(in, "...", "parity:upstream")` for mandatory flags; add `tide.DefaultUpstreamProxyListen = "127.0.0.1:8425"` beside `DefaultCentrifugoListen` (centrifugo.go:19). Docs tree test `TestDocsCommandsMirrorGeneratedMain` requires `docs/` to list the command.
|
||||||
|
|
||||||
|
### `modules/tide/upstream*.go`
|
||||||
|
**Analog for the fake:** `CentrifugoRecorder` (centrifugo.go:30-80): options struct, mutex-guarded slice, `ServeHTTP` with `io.LimitReader(r.Body, Max+1)`, auth header compared and never stored (`Authorization bool`). Mirror that "never keep the secret" rule for masked auth headers in sidecars.
|
||||||
|
**Analog for the proxy:** `Proxy`/`ProxyConfig`/`NewProxy` (proxy.go:42-116): loopback-only validation via `requireLoopbackAddr`, `Fixtures` required, `OpenStore(cfg.VarsPath)`, `varsOutsideFixtures`, `sessions` map under `mu`.
|
||||||
|
|
||||||
|
### `modules/beachcomber` `IndexDropper`
|
||||||
|
Copy the optional-interface pattern exactly (searchable.go:87-108):
|
||||||
|
```go
|
||||||
|
type PageSearcher interface {
|
||||||
|
SearchPage(ctx context.Context, index string, q Query) (SearchResult, error)
|
||||||
|
}
|
||||||
|
func SearchPage(ctx context.Context, e Engine, index string, q Query) (SearchResult, error) {
|
||||||
|
if ps, ok := e.(PageSearcher); ok { return ps.SearchPage(ctx, index, q) }
|
||||||
|
...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Typesense impl from `Flush` (typesense/engine.go:205-215): `e.do(ctx, http.MethodDelete, "/collections/"+url.PathEscape(index), ...)`; return `existed = code != 404`.
|
||||||
|
|
||||||
|
### Redacting slog handler — no analog
|
||||||
|
Use RESEARCH "Redacting handler skeleton". Module README must follow CLAUDE.md structure; add root `README.md` modules-table row (format: README.md lines 86-103). Consumers already resolve `app.Lookup[*slog.Logger]()` then `slog.Default()` (e.g. `FON/jobs.go` `log, _ := p.app.Lookup[*slog.Logger]()`), so install via `slog.SetDefault` in entry points.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan 14-02 — Discogs core, CSV/digest jobs, commands (fonoteka.go)
|
||||||
|
|
||||||
|
### `classes/discogs/client.go`, `rate_limiter.go`
|
||||||
|
**Analog:** `FON/classes/cover_importer.go`.
|
||||||
|
Config-key constants + FromConfig defaults (lines 18-62):
|
||||||
|
```go
|
||||||
|
const (
|
||||||
|
ConfigDiscogsMaxCovers = "golem15.fonoteka.discogs.max_covers"
|
||||||
|
...
|
||||||
|
)
|
||||||
|
func CoverImporterFromConfig(cfg *compass.Config) *CoverImporter {
|
||||||
|
ci := &CoverImporter{MaxCovers: 5, MaxBytes: 10485760, Timeout: 10 * time.Second, HostSuffix: ".discogs.com"}
|
||||||
|
if cfg == nil { return ci }
|
||||||
|
if v := cfg.Int(ConfigDiscogsCoverTimeout); v > 0 { ci.Timeout = time.Duration(v) * time.Second }
|
||||||
|
...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Injectable fetch field (`Fetch CoverFetchFunc`, nil = fetchguard) is the precedent for the client's test transport. Keep `base_uri` a code constant. Injected clock: precedent `NewPubfailCounter(now func() time.Time)` (`classes/public_share.go:54-61`); extend to the `Clock{Now, Sleep(ctx,d)}` interface from RESEARCH Pattern 2.
|
||||||
|
|
||||||
|
### Install real `ReleaseFetcher` (D-08)
|
||||||
|
Seam (csv_import_service.go:589-628): `SetReleaseFetcher(f) (restore func())` with atomic box. Call it from `Plugin.Boot`; tests use the returned `restore`.
|
||||||
|
|
||||||
|
### WR-02 locks in `UpdateCsvMapping` / `UpdateCsvRow` / `CancelCsvImport`
|
||||||
|
**Reference CAS:** `CommitCsvImport` (csv_import_service.go:860-889):
|
||||||
|
```go
|
||||||
|
err := lagoon.Transaction(ctx, db, func(ctx context.Context, tx *gorm.DB) error {
|
||||||
|
res := tx.WithContext(ctx).Exec(`UPDATE golem15_fonoteka_csv_imports SET status = ?, import_mode = ?, updated_at = NOW() WHERE id = ? AND status = ?`,
|
||||||
|
CsvImportStatusImporting, m, imp.ID, CsvImportStatusPreview)
|
||||||
|
...
|
||||||
|
id, err := jobs.Dispatch(ctx, jobDB(tx, ctx), CsvImportArgs{CsvImportID: imp.ID}, conga.DispatchOpts{
|
||||||
|
Label: CsvImportLabel, Queue: CsvImportQueue, Count: int(imp.RowCount), Metadata: csvJobMetadata(imp),
|
||||||
|
})
|
||||||
|
...
|
||||||
|
})
|
||||||
|
```
|
||||||
|
Current unlocked `CancelCsvImport` (lines 891-905) reads `imp.MatchJobID`/`ImportJobID` from the stale struct — replace with `tx.Clauses(clause.Locking{Strength: "UPDATE"})` re-read inside `lagoon.Transaction`. Discogs fetch stays outside the lock.
|
||||||
|
|
||||||
|
### `jobs.go` — three workers
|
||||||
|
**Analog:** `FON/jobs.go:28-41` and the per-run service resolution (lines 45-61):
|
||||||
|
```go
|
||||||
|
func (p *Plugin) Jobs() []pact.Job {
|
||||||
|
return []pact.Job{
|
||||||
|
conga.Job(p.sendInvitationMail,
|
||||||
|
conga.OnQueue(classes.InvitationMailQueue),
|
||||||
|
conga.MaxAttempts(classes.InvitationMailAttempts)),
|
||||||
|
...
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (p *Plugin) sendWishlistPurchasedMail(ctx context.Context, args classes.WishlistPurchasedMailArgs) error {
|
||||||
|
if p.app == nil { return errDatabaseUnavailable }
|
||||||
|
gdb, ok := p.app.Lookup[*gorm.DB]()
|
||||||
|
if !ok || gdb == nil { return errDatabaseUnavailable }
|
||||||
|
mailer, ok := p.app.Lookup[postcard.Mailer]()
|
||||||
|
...
|
||||||
|
log, _ := p.app.Lookup[*slog.Logger]()
|
||||||
|
return deliverWishlistPurchasedMail(ctx, gdb, mailer, log, args)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Split each worker into a thin `p.xxx` resolver plus a testable `deliverXxx(ctx, gdb, ...)` free function, as done here. Add `conga.Timeout(240*time.Second)` on the match job (RESEARCH Pattern 1). Digest mail locale pick = lines 172-175 (`PreferredLocale == "en"` → `-en` template). Update the "no worker until Phase 14" comment (line 30). Logging rule: ids only, never args.
|
||||||
|
|
||||||
|
### `console/prune_notifications.go`, `console/reindex.go`
|
||||||
|
**Analog:** `FON/console/oauth_client.go:28-60`:
|
||||||
|
```go
|
||||||
|
func OAuthClientCommand(app *backpack.App) bonfire.Command {
|
||||||
|
return bonfire.Command{
|
||||||
|
Name: "fonoteka:oauth-client",
|
||||||
|
Description: "...",
|
||||||
|
Flags: []bonfire.Flag{ {Name: "list", Bare: true, Description: "..."} },
|
||||||
|
Run: func(ctx context.Context, in bonfire.Input, out bonfire.Output) error {
|
||||||
|
if app == nil { return errors.New("fonoteka:oauth-client: app is nil") }
|
||||||
|
gdb, ok := app.Lookup[*gorm.DB]()
|
||||||
|
if !ok || gdb == nil { return errors.New("fonoteka:oauth-client: database is not configured") }
|
||||||
|
if listVal, _ := in.Flag("list"); listVal == "true" { ... }
|
||||||
|
...
|
||||||
|
out.Error("Client name is required.")
|
||||||
|
return errors.New("...")
|
||||||
|
```
|
||||||
|
`--drop-old-items-index` is a `Bare: true` flag. Failure = `out.Error(msg)` + return error. Register in `plugin.go:267-272`:
|
||||||
|
```go
|
||||||
|
cmds := []bonfire.Command{console.OAuthClientCommand(p.app)}
|
||||||
|
```
|
||||||
|
Reindex gate: `settingsGate` in `FON/search.go:19-47`; `beachcomber.From(app)` → `Engine()`, `beachcomber.SearchPage`.
|
||||||
|
|
||||||
|
### `updates/<ts>_discogs_rate_windows.go`
|
||||||
|
**Analog:** `FON/updates/11_secrets_slice.go:22-45`:
|
||||||
|
```go
|
||||||
|
apiTokenMigration = &gormigrate.Migration{
|
||||||
|
ID: "202609180009_create_api_tokens",
|
||||||
|
Migrate: func(tx *gorm.DB) error { return tx.Exec(`CREATE TABLE ...`).Error },
|
||||||
|
Rollback: func(tx *gorm.DB) error { return tx.Exec(`DROP TABLE IF EXISTS ...`).Error },
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Register through `updates.Register(...)` (updates/registry.go) in an `init()`. Add the table to `APP/parity/schema_diff_test.go` Go-only allow-list.
|
||||||
|
|
||||||
|
### `parity/discogs_truth_tables.php`
|
||||||
|
**Analog:** `APP/parity/csv_truth_tables.php:1-40` — `PHP_ROOT` env, `require_once` the classes directly, stub `ApplicationException`, `write_table($dir, $name, $data)` with `JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_PRESERVE_ZERO_FRACTION`, output into `classes/discogs/testdata/php_*.json`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan 14-03 — Discogs routes
|
||||||
|
|
||||||
|
### Controllers (`controllers/api/*_controller.go`)
|
||||||
|
**Analog:** `FON/controllers/api/credentials_controller.go`.
|
||||||
|
Handler-factory shape and scope (lines 184-190):
|
||||||
|
```go
|
||||||
|
func DiscogsCredentialStore(app *backpack.App) http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
gdb, user, _, ok := requestScope(w, r, app)
|
||||||
|
if !ok { return }
|
||||||
|
input, ok := readInput(w, r)
|
||||||
|
if !ok { return }
|
||||||
|
```
|
||||||
|
Error writers (all in package `api`): `writeJSON(w, status, v)`, `writeOpaque500(w)`, `writeWinterHTTPError(w, app, http.StatusInternalServerError)` (Winter 500 page — required for SSRF guard failures, D-20), `writeValidationFailed(w, errs, order)` (http_errors.go:78), `marshalNoEscape`. Validation-to-500 helper `validateOr500` (lines 274-287). Translator lookup: `tr, _ := app.Lookup[*phrasebook.Translator]()`. Typed response structs with PHP key order (lines 44-56) instead of maps.
|
||||||
|
|
||||||
|
Inbound limiters: no app controller uses `surf.MemoryStore` yet; construct one per plugin (`surf.NewMemoryStore(sweep)`, `FW/modules/surf/limiter_store.go:32`) and hold it on `Plugin` like `pubfail` (passed into handlers at `routes.go:326`: `api.PublicResolve(p.app, pubfail, "collection")`). Body per RESEARCH "Inbound limiter" example.
|
||||||
|
|
||||||
|
### `routes.go`
|
||||||
|
JWT group (routes.go:45) and token group (routes.go:240):
|
||||||
|
```go
|
||||||
|
r.Group("/_fonoteka/api/v1", surf.Use("jwt.auth", "locale.from-principal", "inv.must-change-password"), func(g pact.Router) {
|
||||||
|
g.Post("/discogs-credential", api.DiscogsCredentialStore(p.app))
|
||||||
|
g.Post("/albums/{id}/photos", albumPhotoUpload, "throttle:20,1")
|
||||||
|
r.Group("/api/v1/fonoteka", surf.Use("inv_token", "throttle:fonoteka-api-token"), func(g pact.Router) {
|
||||||
|
```
|
||||||
|
Token cover-price route gets `"throttle:12,1"` per-route; `inv.scope:*` middleware as on existing token routes. Remove entries from `phase14Absent` in `routes_table_phase13_test.go:270-282` as routes land.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan 14-04 — sm-golem-plugin + AI recognition
|
||||||
|
|
||||||
|
### `golem/go.mod`
|
||||||
|
**Analog:** `USR/go.mod`: `module git.golem15.com/golem15/sm-golem-plugin`, `go 1.27.0`, require `git.golem15.com/golem15/summercms v0.0.0`, and
|
||||||
|
```
|
||||||
|
replace git.golem15.com/golem15/summercms => ../../../../summercms.go
|
||||||
|
```
|
||||||
|
|
||||||
|
### `golem/plugin.go`
|
||||||
|
**Analog:** `USR/plugin.go:1-90`: interface assertions block
|
||||||
|
```go
|
||||||
|
var (
|
||||||
|
_ party.Plugin = (*Plugin)(nil)
|
||||||
|
_ pact.HasConfig = (*Plugin)(nil)
|
||||||
|
_ pact.HasMigrations = (*Plugin)(nil)
|
||||||
|
_ pact.HasModels = (*Plugin)(nil)
|
||||||
|
_ pact.HasLang = (*Plugin)(nil)
|
||||||
|
)
|
||||||
|
//go:embed config
|
||||||
|
var configFS embed.FS
|
||||||
|
//go:embed lang
|
||||||
|
var langFS embed.FS
|
||||||
|
type Plugin struct { app *backpack.App }
|
||||||
|
func (p *Plugin) ID() string { return "golem15.golem" }
|
||||||
|
func (p *Plugin) Requires() []string { return nil }
|
||||||
|
func (p *Plugin) Register(app *backpack.App) error { p.app = app; return nil }
|
||||||
|
```
|
||||||
|
Admin list/form: `FON/admin.go` (`//go:embed` of `controllers/<x>/config_*.yaml` + `models/<x>/{fields,columns}.yaml`, `AdminFS()`, `AdminControllers()` with lazy `db func() *gorm.DB`) and `FON/controllers/genres_admin_controller.go`:
|
||||||
|
```go
|
||||||
|
type genresAdminController struct{}
|
||||||
|
func (genresAdminController) ID() string { return "golem15.fonoteka.genres" }
|
||||||
|
func (genresAdminController) ModelName() string { return `Golem15\Fonoteka\Models\Genre` }
|
||||||
|
func (genresAdminController) ConfigDir() string { return "controllers/genres" }
|
||||||
|
func (genresAdminController) RequiredPermissions() []string { return []string{"golem15.fonoteka.access_genres"} }
|
||||||
|
func (genresAdminController) NewRecord() any { return &models.Genre{} }
|
||||||
|
```
|
||||||
|
`config_list.yaml` template: `FON/controllers/genres/config_list.yaml`. cabana rejects `repeater` — rows are records, not a repeater.
|
||||||
|
|
||||||
|
### `golem/models/ai_model.go`
|
||||||
|
**Analog:** `FON/models/user_ai_credential.go`:
|
||||||
|
```go
|
||||||
|
type UserAiCredential struct {
|
||||||
|
ID uint `gorm:"column:id;primaryKey"`
|
||||||
|
APIKey lagoon.Encrypted `gorm:"column:api_key" json:"-"`
|
||||||
|
BaseURL *string `gorm:"column:base_url"`
|
||||||
|
CreatedAt time.Time `gorm:"column:created_at"`
|
||||||
|
UpdatedAt time.Time `gorm:"column:updated_at"`
|
||||||
|
}
|
||||||
|
func (UserAiCredential) TableName() string { return "golem15_fonoteka_user_ai_credentials" }
|
||||||
|
func (UserAiCredential) Fillable() []string { ... }
|
||||||
|
func (UserAiCredential) Hidden() []string { return []string{"api_key"} }
|
||||||
|
func init() { Register(UserAiCredential{}) }
|
||||||
|
```
|
||||||
|
Plus `models/registry.go` (`Register`/`All`) copied from `FON/models/registry.go`. Keep `models/` a leaf package.
|
||||||
|
|
||||||
|
### AdminVisionModel + trust marker
|
||||||
|
Seam `FON/classes/ai_config_resolver.go:30-45`:
|
||||||
|
```go
|
||||||
|
type AIConfig struct {
|
||||||
|
Adapter string `json:"-"`
|
||||||
|
APIKey string `json:"-"`
|
||||||
|
BaseURL string `json:"-"`
|
||||||
|
Model string `json:"-"`
|
||||||
|
}
|
||||||
|
var AdminVisionModel = func(ctx context.Context) (*AIConfig, error) { return nil, nil }
|
||||||
|
```
|
||||||
|
Add `Trusted bool \`json:"-"\``. Assign `AdminVisionModel` from fonoteka `Boot` (fonoteka requires `golem15.golem`; update `Requires()` at `FON/plugin.go:78`) using a set/restore helper modelled on `SetReleaseFetcher` for test isolation.
|
||||||
|
|
||||||
|
### `golem/classes/ssrf_guard.go`
|
||||||
|
Host matching: reuse semantics of `cover_importer.go` `allowedURL` (lines 66-78) but PHP rule (leading `.` = suffix, else exact, lowercase). Failure surfaces as `writeWinterHTTPError(..., 500)` in callers.
|
||||||
|
|
||||||
|
### Recognize + ai-credential/test controllers
|
||||||
|
As Plan 14-03 controller pattern. Multipart photo intake: `FON/controllers/api/album_photos_controller.go:61` (`AlbumPhotoUpload`) and `classes/image_guard.go`. Tracklist: `classes/tracklist_text_parser.go`.
|
||||||
|
|
||||||
|
### App wiring (go.work / go.mod / summer.yaml / plugins.gen.go / .gitmodules)
|
||||||
|
Current state to extend:
|
||||||
|
```
|
||||||
|
use ( . ./plugins/golem15/user ./plugins/golem15/fonoteka ) # go.work
|
||||||
|
plugins: # summer.yaml
|
||||||
|
- id: golem15.user module: git.golem15.com/golem15/sm-user-plugin
|
||||||
|
- id: golem15.fonoteka module: git.golem15.com/golem15/fonoteka/plugins/golem15/fonoteka
|
||||||
|
[submodule "plugins/golem15/user"] path/url # .gitmodules
|
||||||
|
```
|
||||||
|
Insert `golem15.golem` (and later `golem15.feedback`) before `golem15.fonoteka`; regenerate `plugins.gen.go` with `summer build` (never hand-edit: "Code generated by summer build. DO NOT EDIT."). Submodule commits first, pointer bump separately.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan 14-05 — sm-feedback-plugin
|
||||||
|
|
||||||
|
### `feedback/plugin.go`, `routes.go`
|
||||||
|
Plugin root as Plan 14-04 (`USR/plugin.go`), `go.mod` also `replace git.golem15.com/golem15/sm-user-plugin => ../user`. Routes analog `USR/routes.go`:
|
||||||
|
```go
|
||||||
|
func (p *Plugin) Routes(r pact.Router) error {
|
||||||
|
r.Group("/_user/api/v1", surf.Use("throttle:user-api"), func(g pact.Router) {
|
||||||
|
g.Post("/login", controllers.Login(p.app))
|
||||||
|
})
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Named throttles `feedback-config` / `feedback-submit` via `surf.BucketProvider` `Buckets()` (`USR/plugin.go` `_ surf.BucketProvider = (*Plugin)(nil)` and `func (p *Plugin) Buckets() map[string]surf.Bucket`, using `surf.TrustedProxies(p.app.Config)` for IP keys).
|
||||||
|
|
||||||
|
### GetApiArray listener
|
||||||
|
**Analog:** `FON/plugin.go:92-103`:
|
||||||
|
```go
|
||||||
|
app.Events.Listen[*userclasses.GetApiArrayEvent]("golem15.fonoteka", func(_ context.Context, e *userclasses.GetApiArrayEvent) error {
|
||||||
|
if e == nil || e.User == nil { return nil }
|
||||||
|
m := e.Collected()
|
||||||
|
m["organisation_id"] = e.User.OrganisationID
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
```
|
||||||
|
Use listener id `"golem15.feedback"`; key `feedback_widget_hidden`.
|
||||||
|
|
||||||
|
### Settings singleton + admin screen
|
||||||
|
**Analogs:** `FON/models/settings.go` (typed singleton table, `Fillable`, `Rules`) and `FON/admin_settings.go`:
|
||||||
|
```go
|
||||||
|
func (*Plugin) Settings() []pact.SettingsItem {
|
||||||
|
return []pact.SettingsItem{{
|
||||||
|
Code: "fonoteka", Label: "...", Category: "...", Icon: "search",
|
||||||
|
Model: `Golem15\Fonoteka\Models\Settings`, Order: 500,
|
||||||
|
Permissions: []string{"golem15.fonoteka.manage_settings"},
|
||||||
|
Form: "models/settings/fields.yaml",
|
||||||
|
NewModel: func() any { return &models.Settings{} },
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Use `text` fields instead of `colorpicker`/`readOnly` (cabana rejects unknown keys).
|
||||||
|
|
||||||
|
### G15Office sync job
|
||||||
|
`conga.Job(p.syncG15Office, conga.OnQueue(...), conga.MaxAttempts(3))` per `FON/jobs.go:32-40`; HTTP via new `fetchguard` client (Plan 14-01). Unlike CSV jobs, this one returns the error to retry (PHP rethrows).
|
||||||
|
|
||||||
|
### Image guard
|
||||||
|
Copy `FON/classes/image_guard.go` (`IsAllowedImage`, `SniffImageMIME`) into `feedback/classes/` (per-plugin copies are intentional).
|
||||||
|
|
||||||
|
### Parity corpus
|
||||||
|
`APP/parity/check_corpus.go:21-45` — add a `feedbackRouteIDs` slice beside `userAPIRouteIDs`/`realtimeRouteIDs` and include it in `comparePHPSnapshot` (line 625 `combined := append(...)`); add lines to `routes.snapshot`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan 14-06 — Unit tests and gate
|
||||||
|
|
||||||
|
### `scripts/check-phase14.sh`
|
||||||
|
**Analog:** `FW/scripts/check-phase13.sh` (lines 1-60): `set -euo pipefail`, `unset FORCE_COLOR`, `ROOT`/`APP`/`PHASE_DIR` env overrides (rename `PHASE13_*` → `PHASE14_*`), `APP_PLUGINS=(...)` (add `./plugins/golem15/golem/... ./plugins/golem15/feedback/...`), `EXPECTED_PORTED`/`EXPECTED_PENDING`, `COVERAGE_FLOOR=80`, `usage()` with `--self-test --go --parity --named --coverage --evidence --all`, python `go test -json` detector (exit codes 1-6).
|
||||||
|
|
||||||
|
### Worker tests
|
||||||
|
**Analog:** `FON/job_contract_worker_test.go:19-60` — `bootDB(t)`, `lagoon.Use`, `backpack.New(cfg)`, `lagoon.Publish`, `party.Activate(application, []string{"golem15.user", "golem15.fonoteka"})` (add `golem15.golem` once required), `lagoon.Migrate`, `conga.StartWorker`, `conga.From(application)`. The current assertion that CSV/digest queues are unserved (lines 50-53) must be inverted in Plan 14-02, not 14-06.
|
||||||
|
|
||||||
|
### Clock / limiter tests
|
||||||
|
`NewPubfailCounter(now)` injected-clock precedent; testcontainers Postgres for the UNLOGGED upsert (`FON/classes/postgres_test.go` harness).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Shared Patterns
|
||||||
|
|
||||||
|
### Handler factory + scope
|
||||||
|
**Source:** `FON/controllers/api/request.go:174` `requestScope(w, r, app) (*gorm.DB, *usermodels.User, *models.ApiToken, bool)`; all handlers are `func X(app *backpack.App) http.HandlerFunc`.
|
||||||
|
|
||||||
|
### Error responses
|
||||||
|
**Source:** `FON/controllers/api/http_errors.go` — `writeWinterHTTPError` (Winter HTML pages: `winter_500.html` etc.), `writeValidationFailed`, `marshalNoEscape`; `writeOpaque500`. Plugins in their own repos cannot import fonoteka's `api` package; feedback/golem need their own copies (user plugin precedent: `USR/controllers/winter_error_page.html`).
|
||||||
|
|
||||||
|
### Secrets
|
||||||
|
`lagoon.Encrypted` + `json:"-"` + `Hidden()`; never log args, tokens or bodies (jobs.go docs: "The args, the token and the URL are never logged"); log ids with `slog.Uint64(...)`.
|
||||||
|
|
||||||
|
### Service lookup
|
||||||
|
`app.Lookup[*gorm.DB]()`, `app.Lookup[postcard.Mailer]()`, `app.Lookup[*slog.Logger]()` resolved per call/run, never cached at Register (Boot runs before serve publishes the DB).
|
||||||
|
|
||||||
|
### Registries
|
||||||
|
`models.Register` / `updates.Register` in `init()` (`FON/models/registry.go`, `FON/updates/registry.go`).
|
||||||
|
|
||||||
|
### Transactions / CAS
|
||||||
|
`lagoon.Transaction(ctx, db, func(ctx, tx) error {...})` + `jobs.Dispatch(ctx, jobDB(tx, ctx), args, conga.DispatchOpts{...})` inside the tx (csv_import_service.go:860-889).
|
||||||
|
|
||||||
|
### Framework docs rule
|
||||||
|
Any exported change in `modules/fetchguard`, `tide`, `beachcomber`, new slog module → README + `docs/` same commit; verify with `go test ./cmd/summer -run TestDocsTree` and `go run ./cmd/summer docs:build --check`.
|
||||||
|
|
||||||
|
## No Analog Found
|
||||||
|
|
||||||
|
| File | Role | Data Flow | Reason |
|
||||||
|
|---|---|---|---|
|
||||||
|
| redacting slog handler module | log middleware | transform | No slog.Handler wrapper exists; use RESEARCH skeleton |
|
||||||
|
| `golem/classes/providers/{anthropic,openai}.go` | adapter | request-response | No outbound JSON POST adapters exist yet (first users of the new fetchguard client) |
|
||||||
|
| tide MITM CONNECT proxy (TLS termination, local CA) | utility | streaming | `tide/proxy.go` is a plain loopback reverse proxy; CA/CONNECT handling is new |
|
||||||
|
| Discogs UNLOGGED atomic upsert limiter | service | CRUD | No cross-process limiter exists (`PubfailCounter` is in-memory); SQL from RESEARCH D-17 |
|
||||||
|
| `golem` system_settings importer | migration | batch | No prior importer from Winter `system_settings`; conditional on table existence |
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
**Analog search scope:** `FW/modules/{fetchguard,tide,beachcomber,surf}`, `FW/cmd/summer`, `FW/scripts`, `FON/{classes,controllers,console,models,updates,jobs.go,plugin.go,routes.go,admin*.go,schedule.go}`, `USR/{plugin.go,routes.go,go.mod}`, `APP/{parity,go.work,summer.yaml,plugins.gen.go,.gitmodules}`
|
||||||
|
**Files scanned:** ~40
|
||||||
|
**Pattern extraction date:** 2026-10-03
|
||||||
Reference in New Issue
Block a user