docs(14-04): complete sm-golem-plugin and AI recognition plan

This commit is contained in:
Jakub Zych
2026-10-03 23:39:07 +02:00
parent 7c2c43359f
commit b3e4626d2d

View File

@@ -0,0 +1,298 @@
---
phase: 14-domain-jobs-and-external-integrations
plan: 04
subsystem: ai
tags: [sm-golem-plugin, golem15.golem, anthropic, openai, ai-service, ssrf, fetchguard, cabana, recognition, parity, upstream-sidecar, submodule]
requires:
- phase: 14-domain-jobs-and-external-integrations
provides: "14-01 fetchguard.Client (TrustedMode, PublicOnlyMode, WithTransport), tide upstream sidecars and summer parity:upstream; 14-03 InboundLimits, phase14Routes and the per-case recording loop"
- phase: 13-p-ytarium-api-wishlist-notifications-csv-credentials-public
provides: "ResolveAIConfig, AIAllowed, the AdminVisionModel seam and the BYOK credential tables"
provides:
- "New shared core plugin repo git.golem15.com/golem15/sm-golem-plugin (master pushed, cd3f0f6..c93e2d9), mounted in fonoteka.go at plugins/golem15/golem: plugin golem15.golem, package golem"
- "golem AI layer: Prompt/Payload/AIResponse, OpenAI-compatible and Anthropic adapters over fetchguard (no SDK), AIService (Send, SendStream, SendFile, SendFilePath, SendFileReader, SendWithModel, SendToImageModel, SendToVisionModel, GenerateImage, Ask), PromptFactory"
- "golem15_golem_models (encrypted write-only api_key) with its cabana list and form under golem15.golem.access_settings, the Settings lookups and golem:import-settings"
- "security.AssertSafeURL / AllowedHosts / UnsafeURLError (SSRFGuard port, GOLEM15_SSRF_ALLOWED_HOSTS)"
- "fetchguard.IsPrivateAddr (framework)"
- "fonoteka: AIConfig.Trusted, SetAdminVisionModel/AdminVisionModel(ctx, db) fed by the golem vision model, UserAIConfig, guarded base_url overrides, RecognizeAlbums, AICredentialTest and AlbumRecognize on both groups"
- "Parity: 20 PHP-recorded cases (7 ai-credential/test, 10 recognize JWT, 3 recognize token) with 12 upstream sidecars; ported 168, pending 3 (D-09)"
affects: [14-05 sm-feedback-plugin (same submodule workflow), 14-06 unit tests and gate, 15 cutover (GOLEM15_SSRF_ALLOWED_HOSTS, golem:import-settings)]
actuals:
tokens: 90000 # chars/4 over the added lines of all three repos
tasks: 4
commits: 1 # MEASURED in summercms.go: plan_head_before..plan_head_after (code only)
app_repo_commits: 8 # MEASURED in fonoteka.go: app_repo_head_before..app_repo_head_after
plugin_repo_commits: 3 # MEASURED in sm-golem-plugin: the whole new repo (pushed)
plan_head_before: f519261da6fe62b7dc07ce842f4df8f4428ad14c
plan_head_after: 7c2c43359fd460384848bb9cd9916906d2ef3cbc
app_repo_head_before: 28349433d77ba16c0dec7c87e41ef486996169f2
app_repo_head_after: eea0b1607ef9b5d14b3a5186bf322e6eb45a4b9b
plugin_repo_head_after: c93e2d9
tech-stack:
added: []
patterns:
- "A shared core plugin is born as an empty clone at its mount path, committed and pushed there, then registered with git submodule add; the application commits .gitmodules plus the gitlink alone, and later bumps as gitlink-only commits"
- "Application tests that would change with a submodule bump are made bump-neutral first (filter the plugin's own history, tables and menu), so the bump commit stays a pure gitlink change and green"
- "AI calls go through one AIService per app (services.ForApp); Trusted configs use fetchguard TrustedMode, every user or org config PublicOnlyMode after security.AssertSafeURL"
- "PHP's raw-curl requests carry Accept: */* and no User-Agent; Go sends the same (an empty User-Agent value keeps net/http from adding its own)"
- "Stateless app seams take the request's database (AdminVisionModel(ctx, db)), as 14-02's ReleaseFetcher does"
key-files:
created:
- ../fonoteka.go/plugins/golem15/golem/ (whole repo: plugin.go, admin.go, README.md, config, lang, models, updates, controllers, console, classes/{valueobjects,providers,services,security,factories}, internal/pgtest)
- ../fonoteka.go/plugins/golem15/fonoteka/classes/album_recognition.go
- ../fonoteka.go/plugins/golem15/fonoteka/classes/album_recognition_test.go
- ../fonoteka.go/plugins/golem15/fonoteka/controllers/api/recognize_controller.go
- ../fonoteka.go/plugins/golem15/fonoteka/golem_wiring.go
- ../fonoteka.go/plugins/golem15/fonoteka/ai_routes_test.go
- ../fonoteka.go/parity/upstream/scripts/{openai-chat-401,openai-chat-pong,anthropic-messages-pong,openai-recognize,openai-recognize-truncated,openai-recognize-garbage,anthropic-recognize,anthropic-recognize-truncated}.yaml
- ../fonoteka.go/parity/fixtures/routes/files/spoofed.jpg
modified:
- modules/fetchguard/ip.go
- modules/fetchguard/README.md
- docs/services/outbound-http.md
- ../fonoteka.go/.gitmodules
- ../fonoteka.go/go.work
- ../fonoteka.go/go.mod
- ../fonoteka.go/summer.yaml
- ../fonoteka.go/plugins.gen.go
- ../fonoteka.go/app/app.go
- ../fonoteka.go/plugins/golem15/fonoteka/go.mod
- ../fonoteka.go/plugins/golem15/fonoteka/plugin.go
- ../fonoteka.go/plugins/golem15/fonoteka/routes.go
- ../fonoteka.go/plugins/golem15/fonoteka/classes/ai_config_resolver.go
- ../fonoteka.go/plugins/golem15/fonoteka/classes/gates.go
- ../fonoteka.go/plugins/golem15/fonoteka/controllers/api/credentials_controller.go
- ../fonoteka.go/plugins/golem15/fonoteka/controllers/api/inbound_limits.go
- ../fonoteka.go/parity/{manifest.yaml,parity_test.go,parity_contract_test.go,fonoteka_seed_test.go,fonoteka_reset.php,php_parity.sh,check_corpus.go,README.md}
key-decisions:
- "ai-credential/test tests the caller's own stored credential only (PHP UserAiConfig::fromCredential), not the org or admin tier the plan named"
- "AdminVisionModel takes the request's database: AdminVisionModel(ctx, db) and SetAdminVisionModel(func(ctx, db)); Boot installs a stateless reader of the golem vision model, Trusted set by the seam"
- "The api package maps an AIConfig to the golem ModelConfig through (*classes.AIConfig).ModelConfig(), because the api package cannot import the plugin root"
- "fetchguard.IsPrivateAddr is exported so the SSRF guard judges resolved addresses with the dial guard's table"
- "GenerateImage keeps PHP's contract and does not fetch or check the returned URL; callers run security.AssertSafeURL (documented, tested)"
- "An admin model with an empty base_url uses the provider default (PHP would post to /messages and fail to connect)"
- "The admin api_key field is write-only: an update that sends it empty or redacted keeps the stored key"
- "php_parity.sh exports GOLEM15_SSRF_ALLOWED_HOSTS empty so PHP records with the plugin default allowlist, which the Go replay uses"
- "INTG-02 is not marked complete: 14-06 also declares it (requirements.ready-ids gate)"
patterns-established:
- "golem-vision and ai are additive seed extras on both sides, like discogs"
- "Recorded multipart cases copy their part files into the specs directory (the recorder refuses symlinks)"
requirements-completed: [INTG-02]
coverage:
- id: D1
description: "sm-golem-plugin exists, is pushed and mounted; golem15.golem loads before golem15.fonoteka"
requirement: INTG-02
verification:
- kind: unit
ref: "plugins/golem15/golem#TestGolemPluginBoot; plugins/golem15/fonoteka#TestRouteTablePhase12/13/14"
status: pass
- kind: other
ref: "git -C plugins/golem15/golem status --porcelain --branch → '## master...origin/master'; grep checks of .gitmodules, go.mod, summer.yaml, plugins.gen.go"
status: pass
human_judgment: false
- id: D2
description: "OpenAI-compatible and Anthropic adapters and the full AIService with PHP's failure messages"
requirement: INTG-02
verification:
- kind: unit
ref: "classes/providers#TestOpenAIAdapterPayload, TestAnthropicAdapterPayload; classes/services#TestAIServiceFailures, TestAIServiceStream; classes/factories#TestPromptFactory"
status: pass
human_judgment: false
- id: D3
description: "AI models table, admin list and form, settings lookups with the default-model quirk, one-time importer"
requirement: INTG-02
verification:
- kind: unit
ref: "plugins/golem15/golem#TestImportSettings, TestAdminModelsForm; classes/services#TestDefaultModelQuirk"
status: pass
human_judgment: false
- id: D4
description: "SSRF guard port and the trusted/guarded split"
requirement: INTG-02
verification:
- kind: unit
ref: "classes/security#TestSSRFGuard; classes/services#TestAdminModelTrusted; modules/fetchguard#TestIsPrivateAddr, ExampleIsPrivateAddr"
status: pass
human_judgment: false
- id: D5
description: "AI tier resolution (admin vision model trusted, user and org guarded, no fallback) and ai-credential/test"
requirement: INTG-02
verification:
- kind: unit
ref: "plugins/golem15/fonoteka#TestAICredentialTestRoute, TestAdminVisionTier, TestResolveAIConfigPrecedence"
status: pass
- kind: parity
ref: "parity#TestParityCorpus/POST___fonoteka_api_v1_ai-credential_test_jwt (7 cases, 4 sidecars)"
status: pass
human_judgment: false
- id: D6
description: "Album recognition on the JWT and token groups with PHP's answers"
requirement: INTG-02
verification:
- kind: unit
ref: "plugins/golem15/fonoteka/classes#TestRecognizeAlbums; plugins/golem15/fonoteka#TestRecognizeRoutes"
status: pass
- kind: parity
ref: "parity#TestParityCorpus/POST___fonoteka_api_v1_albums_recognize_jwt (10 cases), POST__api_v1_fonoteka_albums_recognize_personal_token (3 cases); TestParityCorpus/coverage 168 ported, 3 pending"
status: pass
human_judgment: false
duration: 69min
completed: 2026-10-03
status: complete
---
# Phase 14 Plan 04: sm-golem-plugin and AI cover recognition Summary
**A new shared plugin, sm-golem-plugin, brings the Golem AI layer to Go: SDK-free OpenAI-compatible and Anthropic adapters through the guarded client, the operator's AI models with an admin screen and a one-time importer, and the exact SSRF guard. On top of it a collector can test an AI key and photograph albums in the Nuxt app or through MCP, and gets PHP's answers, recorded from PHP and replayed offline.**
## Performance
- **Duration:** 69 min
- **Started:** 2026-10-03T20:27:58Z
- **Completed:** 2026-10-03T21:37:08Z
- **Tasks:** 4
- **Commits:** 3 in sm-golem-plugin (pushed), 8 in fonoteka.go, 1 code commit in summercms.go (plus this summary)
- **Files:** 34 in the plugin (+4505), 105 in fonoteka.go (+2664/-176), 5 in summercms.go (+55)
## Accomplishments
- **sm-golem-plugin (D-01).** The empty remote was cloned at `plugins/golem15/golem`, built, committed and pushed (`master` at `c93e2d9`, in sync with origin), then registered with `git submodule add`. fonoteka.go loads `golem15.golem` before `golem15.fonoteka`: `go.work`, both `go.mod` files, `summer.yaml`, the regenerated `plugins.gen.go` and `app.PluginIDs` name it. `golem15.fonoteka` requires it, and every test activation list names it.
- **AI layer (D-02, D-04).**
- `OpenAIAdapter` and `AnthropicAdapter` are hand-written JSON mappings, with no SDK. The OpenAI adapter maps `max_completion_tokens`, `response_format` and `reasoning_effort` as PHP does. The Anthropic adapter sets the `max_tokens` chain, builds image and document blocks and maps usage.
- `AIService` covers every PHP call except faces and chat, with PHP's messages.
- Requests go through `fetchguard`: `TrustedMode` for operator models and `PublicOnlyMode` for everyone else. They carry PHP curl's `Accept: */*` and no User-Agent.
- **Models and admin (D-03, D-18).** `golem15_golem_models` keeps an encrypted, write-only `api_key` and has a cabana list and form under `golem15.golem.access_settings`. Its lookups keep PHP's default-model quirk. `golem:import-settings` copies the `golem_settings` repeater once and encrypts each key.
- **SSRF (D-05, D-20).** `security.AssertSafeURL` ports SSRFGuard: https only, the allowlist (overridable with `GOLEM15_SSRF_ALLOWED_HOSTS`), and the resolve-time check through the new `fetchguard.IsPrivateAddr`. User and org `base_url` overrides are checked when they are resolved. A refused URL answers Winter's 500 page.
- **AI tier (D-03).** `AIConfig.Trusted` is set only by the admin seam, which Boot fills from the golem vision model. A normal user never gets the admin model, and an org-locked member never falls back to it.
- **Routes (INTG-02, D-07).**
- `POST ai-credential/test` (JWT) tests an inline key or the caller's own stored key.
- `POST albums/recognize` runs on the JWT group and on the token group (`inv.scope:ai`). The checks run in PHP's order: gate, limiter (10 per 60 s), validation, image guard, then the AI tier. It answers 200 with the albums, 200 with `recognition_truncated`, or 502.
- `RecognizeAlbums` sends PHP's system prompt verbatim, with the collection's genre hint and the language directive. It retries once, then normalises the answer as PHP does.
- **Parity.** 20 cases were recorded against PHP through `summer parity:upstream`, 12 of them with sidecars, covering both the OpenAI and the Anthropic exchanges. The cases include the admin tier through the global vision model, truncation on `length` and on `max_tokens`, an empty answer, a provider error, a JPEG polyglot refused before any upstream call, and the unsafe and non-allowlisted base URLs (500 page, no upstream). The corpus has 168 routes ported and passing and 3 pending (the D-09 routes). `check_corpus --require-recorded --check-secrets` is green. A mutation of one word in the prompt failed both recognize routes, which shows the upstream body is asserted.
## Task Commits
sm-golem-plugin (`git@git.golem15.com:golem15/sm-golem-plugin.git`, master, pushed):
1. `cd3f0f6` feat: golem15.golem plugin with the OpenAI-compatible adapter and AIService.Send (Task 1)
2. `da9fcde` feat: security.AssertSafeURL ports the SSRFGuard (Task 2)
3. `c93e2d9` feat: AI models admin, settings import, Anthropic adapter and the rest of AIService (Task 2)
fonoteka.go (not pushed):
1. `91b3f72` chore(14-04): mount sm-golem-plugin at plugins/golem15/golem (`.gitmodules` + gitlink only)
2. `bf90131` feat(14-04): the application loads golem15.golem before golem15.fonoteka (Task 1)
3. `cf18030` fix(14-04): check_corpus header scans stay on one line (Task 1 blocker)
4. `bbb641b` feat(14-04): a collector tests an AI key and gets PHP's answer through golem15.golem (Task 1)
5. `cce718a` test(14-04): fonoteka's migration, schema and admin menu checks ignore golem15.golem's own (Task 2)
6. `8ec5e34` chore(14-04): bump sm-golem-plugin (gitlink only, Task 2)
7. `72b6edc` feat(14-04): the AI tier is chosen as PHP chooses it and unsafe base URLs get PHP's 500 page (Task 3)
8. `eea0b16` feat(14-04): a collector photographs albums in the Nuxt app or through MCP and gets PHP's recognised album list (Task 4)
summercms.go:
- `7c2c433` feat(14-04): fetchguard.IsPrivateAddr exposes the dial guard's address classification (README and docs updated; TestDocsTree and docs:build --check pass)
Tracer gate: after Task 1 the `<verify>` was re-run end to end (vet, the named tests with `-race`, the corpus, check_corpus, submodule status) and passed. The expansion tasks then went ahead.
## Deviations from Plan
### Auto-fixed issues
**1. [Rule 3 - Blocking] `fetchguard.IsPrivateAddr` exported (framework)**
- **Issue:** The plan's guard must use "fetchguard's classification", but `isReservedOrPrivate` was unexported, and a plugin cannot reach it.
- **Fix:** Exported `IsPrivateAddr`, with a test, an example, README and `docs/services/outbound-http.md`.
- **Commit:** 7c2c433 (summercms.go)
**2. [Rule 1 - Bug] check_corpus flagged masked credential headers**
- **Issue:** Once its `{{secret:ai-key}}` placeholder is stripped, `Authorization: Bearer` ran on through `\s+` into the next sorted header (`Content-Type:`). The same would happen to an unquoted `X-Api-Key`.
- **Fix:** The patterns stay on one line (`[ \t]`). Two clean vectors were added to the test.
- **Commit:** cf18030
**3. [Rule 3 - Blocking] The parity mismatch probe used ai-credential/test**
- **Fix:** `assertPortedMismatch` now probes `GET oauth-identities`, a D-09 route that stays pending.
- **Commit:** bbb641b
**4. [Rule 3 - Blocking] Tests that pinned the plugin list**
- **Issue:** `activateAppPlugins` asserted two plugins. Once golem has migrations, a menu and a table, the migration-status indexes, the history-table list, the schema snapshot and the admin navigation test would all change.
- **Fix:** The activation-order check came first (in bf90131). The other checks were then made bump-neutral in cce718a: `withoutGolem`, the history-table filter, an `allowedDiffs` entry and `fonotekaCodes`. As a result the pointer bump `8ec5e34` changes only the gitlink and stays green, as the acceptance criterion requires.
**5. [Rule 1 - Own omission] The part-file audit (T-12-17) rejected `spoofed.jpg`**
- **Fix:** The audit allows the deliberate polyglot (it must not be an image). The fix was folded into eea0b16 before anything was pushed, so every commit is green.
**6. [Rule 2 - Correctness] Parity recording environment**
- `php_parity.sh` exports `GOLEM15_SSRF_ALLOWED_HOSTS=` (empty), so PHP uses the plugin default whatever the checkout's `.env` says.
- `fonoteka_reset.php` forgets the `golem_settings` query cache before `Settings::set`. Otherwise Winter saves onto the deleted row (the first `admin-truncated` recording answered 403 because of this).
### Plan statements that conflict with PHP (parity kept)
**7.** `ai-credential/test` tests the caller's own stored credential (PHP `UserAiConfig::fromCredential`), not "the stored user or org credential through ResolveAIConfig".
**8.** `GenerateImage` does not run `AssertSafeURL` on the returned URL, because PHP's `generateImage` does not either; its caller does. The README says so, and `TestAIServiceFailures` checks the returned URL through the guard.
### Plan details refined
**9.** `AdminVisionModel(ctx, db)` and `SetAdminVisionModel(func(ctx, db))` replace the db-less signature. The seam is stateless and serves every app in the process, which is the 14-02 ReleaseFetcher precedent. Trusted is set by the seam itself.
**10.** The plan's `aiModelConfig` lives in `classes` as `(*AIConfig).ModelConfig()`, because the api package cannot import the plugin root. `golem_wiring.go` holds the vision-model source.
**11.** `NewAIService(db func() *gorm.DB, cfg)` and `services.ForApp(app)` resolve the database per call. `ModelConfig` also carries `AcceptsImages`, `GeneratesImages` and `Position`, which the lookups and the default-model quirk need. `SendFileReader` uploads from a reader, which is the Go form of PHP's `sendFile(File)`.
**12.** The genre hint is `GROUP BY name ORDER BY MAX(created_at) DESC LIMIT 20`, because Postgres rejects PHP's `DISTINCT` with `ORDER BY` on an unselected column. The recorded prompts match.
**13.** No recorded 429 case for recognize. The Go replay keeps one app, and so one in-memory bucket, per route, so the JWT cases reach the limiter at most nine times. `TestRecognizeRoutes` asserts that the 11th call is 429.
**14.** An admin model with an empty `base_url` uses the provider's default. PHP would build `"/messages"` and fail to connect.
**15.** Extra tests beyond the plan: `TestAdminModelsForm` (write-only key through cabana's CRUD), `TestPromptFactory`, `TestIsPrivateAddr` and `ExampleIsPrivateAddr`.
**16.** Commits land on `master` in both repos, as the sequential-executor instructions and the earlier Phase 14 plans do (`branching_strategy: none`).
---
**Total deviations:** 16. Six are auto-fixes, two are plan conflicts resolved for parity, and eight are refinements. **Impact:** the only framework change is the small, documented `fetchguard.IsPrivateAddr`. There is no scope creep.
## Issues Encountered
- The shell aliases `rm` to its interactive form, and one merge step waited on its prompt. Later steps use `command rm -f`.
- A `pkill -f` whose pattern matched its own command line killed the shell. The PHP server was then stopped by PID.
- The recordings write nothing into the PHP checkout: the AI routes store no files, and `git status` there shows only the user's own pre-existing files.
## Verification
- sm-golem-plugin: `go vet ./...` and `go test ./...` pass, both standalone (`GOWORK=off`) and in the workspace. Every named test passes with `-race`.
- fonoteka.go: `go vet ./...` passes, and `go test ./... ./plugins/golem15/fonoteka/... ./plugins/golem15/golem/... -count=1` passes in every package. `go test ./... -short` passes.
- Parity: `TestParityCorpus` reports 171/171 recorded, 168 passing, 0 failing and 3 pending. `TestCheckCorpusPortedCaseStatus`, `TestUpstreamSidecarsAreReplayed` and `TestParityContract` pass. `check_corpus --require-recorded --check-secrets` passes, with 0 pending case-status mismatches.
- summercms.go: `go vet ./...` and `go test ./... -count=1` pass. `TestDocsTree` and `summer docs:build --check` pass.
- Acceptance greps for all four tasks pass. `expectedPortedRoutes` is 168, `phase14Absent` is gone, and `inv.scope:ai` sits on the recognize line. The pointer-bump commit touches only `plugins/golem15/golem`, and the plugin README names no consuming application.
## Known Stubs
None.
## Threat Flags
None. The new surfaces are in the plan's threat register: the AI models admin (T-14-24, permission `golem15.golem.access_settings`, write-only encrypted key), the user/org base URLs (T-14-22), trusted mode (T-14-23) and the recognize route (T-14-25..27). No key, token or DSN literal was pushed (T-14-29: the diff was scanned before each push; test keys are short fakes).
## User Setup Required
None for development. At cutover (Phase 15) the operator runs `golem:import-settings` once and confirms whether production needs `GOLEM15_SSRF_ALLOWED_HOSTS` (for example `.anthropic.com` for user Anthropic base URLs). This is research Open Question 1, still open.
## Next Phase Readiness
- 14-05 can follow the same submodule bootstrap for sm-feedback-plugin: empty clone at the mount path, push, `git submodule add`, a gitlink-only commit, and bump-neutral app tests first.
- 14-06 picks up the unit-test gate. INTG-02 is left Pending because 14-06 also declares it.
---
*Phase: 14-domain-jobs-and-external-integrations*
*Completed: 2026-10-03*
## Self-Check: PASSED
Every listed file exists; commits 7c2c433 (summercms.go), 91b3f72, bf90131, cf18030, bbb641b, cce718a, 8ec5e34, 72b6edc and eea0b16 (fonoteka.go) and cd3f0f6, da9fcde and c93e2d9 (sm-golem-plugin, on origin/master) are present.