From b3e4626d2df317c1b0297c1d0c2b39ca9bb07ab0 Mon Sep 17 00:00:00 2001 From: Jakub Zych Date: Sat, 3 Oct 2026 23:39:07 +0200 Subject: [PATCH] docs(14-04): complete sm-golem-plugin and AI recognition plan --- .../14-04-SUMMARY.md | 298 ++++++++++++++++++ 1 file changed, 298 insertions(+) create mode 100644 .planning/phases/14-domain-jobs-and-external-integrations/14-04-SUMMARY.md diff --git a/.planning/phases/14-domain-jobs-and-external-integrations/14-04-SUMMARY.md b/.planning/phases/14-domain-jobs-and-external-integrations/14-04-SUMMARY.md new file mode 100644 index 0000000..e38934c --- /dev/null +++ b/.planning/phases/14-domain-jobs-and-external-integrations/14-04-SUMMARY.md @@ -0,0 +1,298 @@ +--- +phase: 14-domain-jobs-and-external-integrations +plan: 04 +subsystem: ai +tags: [sm-golem-plugin, golem15.golem, anthropic, openai, ai-service, ssrf, fetchguard, cabana, recognition, parity, upstream-sidecar, submodule] + +requires: + - phase: 14-domain-jobs-and-external-integrations + provides: "14-01 fetchguard.Client (TrustedMode, PublicOnlyMode, WithTransport), tide upstream sidecars and summer parity:upstream; 14-03 InboundLimits, phase14Routes and the per-case recording loop" + - phase: 13-p-ytarium-api-wishlist-notifications-csv-credentials-public + provides: "ResolveAIConfig, AIAllowed, the AdminVisionModel seam and the BYOK credential tables" +provides: + - "New shared core plugin repo git.golem15.com/golem15/sm-golem-plugin (master pushed, cd3f0f6..c93e2d9), mounted in fonoteka.go at plugins/golem15/golem: plugin golem15.golem, package golem" + - "golem AI layer: Prompt/Payload/AIResponse, OpenAI-compatible and Anthropic adapters over fetchguard (no SDK), AIService (Send, SendStream, SendFile, SendFilePath, SendFileReader, SendWithModel, SendToImageModel, SendToVisionModel, GenerateImage, Ask), PromptFactory" + - "golem15_golem_models (encrypted write-only api_key) with its cabana list and form under golem15.golem.access_settings, the Settings lookups and golem:import-settings" + - "security.AssertSafeURL / AllowedHosts / UnsafeURLError (SSRFGuard port, GOLEM15_SSRF_ALLOWED_HOSTS)" + - "fetchguard.IsPrivateAddr (framework)" + - "fonoteka: AIConfig.Trusted, SetAdminVisionModel/AdminVisionModel(ctx, db) fed by the golem vision model, UserAIConfig, guarded base_url overrides, RecognizeAlbums, AICredentialTest and AlbumRecognize on both groups" + - "Parity: 20 PHP-recorded cases (7 ai-credential/test, 10 recognize JWT, 3 recognize token) with 12 upstream sidecars; ported 168, pending 3 (D-09)" +affects: [14-05 sm-feedback-plugin (same submodule workflow), 14-06 unit tests and gate, 15 cutover (GOLEM15_SSRF_ALLOWED_HOSTS, golem:import-settings)] + +actuals: + tokens: 90000 # chars/4 over the added lines of all three repos + tasks: 4 + commits: 1 # MEASURED in summercms.go: plan_head_before..plan_head_after (code only) + app_repo_commits: 8 # MEASURED in fonoteka.go: app_repo_head_before..app_repo_head_after + plugin_repo_commits: 3 # MEASURED in sm-golem-plugin: the whole new repo (pushed) +plan_head_before: f519261da6fe62b7dc07ce842f4df8f4428ad14c +plan_head_after: 7c2c43359fd460384848bb9cd9916906d2ef3cbc +app_repo_head_before: 28349433d77ba16c0dec7c87e41ef486996169f2 +app_repo_head_after: eea0b1607ef9b5d14b3a5186bf322e6eb45a4b9b +plugin_repo_head_after: c93e2d9 + +tech-stack: + added: [] + patterns: + - "A shared core plugin is born as an empty clone at its mount path, committed and pushed there, then registered with git submodule add; the application commits .gitmodules plus the gitlink alone, and later bumps as gitlink-only commits" + - "Application tests that would change with a submodule bump are made bump-neutral first (filter the plugin's own history, tables and menu), so the bump commit stays a pure gitlink change and green" + - "AI calls go through one AIService per app (services.ForApp); Trusted configs use fetchguard TrustedMode, every user or org config PublicOnlyMode after security.AssertSafeURL" + - "PHP's raw-curl requests carry Accept: */* and no User-Agent; Go sends the same (an empty User-Agent value keeps net/http from adding its own)" + - "Stateless app seams take the request's database (AdminVisionModel(ctx, db)), as 14-02's ReleaseFetcher does" + +key-files: + created: + - ../fonoteka.go/plugins/golem15/golem/ (whole repo: plugin.go, admin.go, README.md, config, lang, models, updates, controllers, console, classes/{valueobjects,providers,services,security,factories}, internal/pgtest) + - ../fonoteka.go/plugins/golem15/fonoteka/classes/album_recognition.go + - ../fonoteka.go/plugins/golem15/fonoteka/classes/album_recognition_test.go + - ../fonoteka.go/plugins/golem15/fonoteka/controllers/api/recognize_controller.go + - ../fonoteka.go/plugins/golem15/fonoteka/golem_wiring.go + - ../fonoteka.go/plugins/golem15/fonoteka/ai_routes_test.go + - ../fonoteka.go/parity/upstream/scripts/{openai-chat-401,openai-chat-pong,anthropic-messages-pong,openai-recognize,openai-recognize-truncated,openai-recognize-garbage,anthropic-recognize,anthropic-recognize-truncated}.yaml + - ../fonoteka.go/parity/fixtures/routes/files/spoofed.jpg + modified: + - modules/fetchguard/ip.go + - modules/fetchguard/README.md + - docs/services/outbound-http.md + - ../fonoteka.go/.gitmodules + - ../fonoteka.go/go.work + - ../fonoteka.go/go.mod + - ../fonoteka.go/summer.yaml + - ../fonoteka.go/plugins.gen.go + - ../fonoteka.go/app/app.go + - ../fonoteka.go/plugins/golem15/fonoteka/go.mod + - ../fonoteka.go/plugins/golem15/fonoteka/plugin.go + - ../fonoteka.go/plugins/golem15/fonoteka/routes.go + - ../fonoteka.go/plugins/golem15/fonoteka/classes/ai_config_resolver.go + - ../fonoteka.go/plugins/golem15/fonoteka/classes/gates.go + - ../fonoteka.go/plugins/golem15/fonoteka/controllers/api/credentials_controller.go + - ../fonoteka.go/plugins/golem15/fonoteka/controllers/api/inbound_limits.go + - ../fonoteka.go/parity/{manifest.yaml,parity_test.go,parity_contract_test.go,fonoteka_seed_test.go,fonoteka_reset.php,php_parity.sh,check_corpus.go,README.md} + +key-decisions: + - "ai-credential/test tests the caller's own stored credential only (PHP UserAiConfig::fromCredential), not the org or admin tier the plan named" + - "AdminVisionModel takes the request's database: AdminVisionModel(ctx, db) and SetAdminVisionModel(func(ctx, db)); Boot installs a stateless reader of the golem vision model, Trusted set by the seam" + - "The api package maps an AIConfig to the golem ModelConfig through (*classes.AIConfig).ModelConfig(), because the api package cannot import the plugin root" + - "fetchguard.IsPrivateAddr is exported so the SSRF guard judges resolved addresses with the dial guard's table" + - "GenerateImage keeps PHP's contract and does not fetch or check the returned URL; callers run security.AssertSafeURL (documented, tested)" + - "An admin model with an empty base_url uses the provider default (PHP would post to /messages and fail to connect)" + - "The admin api_key field is write-only: an update that sends it empty or redacted keeps the stored key" + - "php_parity.sh exports GOLEM15_SSRF_ALLOWED_HOSTS empty so PHP records with the plugin default allowlist, which the Go replay uses" + - "INTG-02 is not marked complete: 14-06 also declares it (requirements.ready-ids gate)" + +patterns-established: + - "golem-vision and ai are additive seed extras on both sides, like discogs" + - "Recorded multipart cases copy their part files into the specs directory (the recorder refuses symlinks)" + +requirements-completed: [INTG-02] + +coverage: + - id: D1 + description: "sm-golem-plugin exists, is pushed and mounted; golem15.golem loads before golem15.fonoteka" + requirement: INTG-02 + verification: + - kind: unit + ref: "plugins/golem15/golem#TestGolemPluginBoot; plugins/golem15/fonoteka#TestRouteTablePhase12/13/14" + status: pass + - kind: other + ref: "git -C plugins/golem15/golem status --porcelain --branch → '## master...origin/master'; grep checks of .gitmodules, go.mod, summer.yaml, plugins.gen.go" + status: pass + human_judgment: false + - id: D2 + description: "OpenAI-compatible and Anthropic adapters and the full AIService with PHP's failure messages" + requirement: INTG-02 + verification: + - kind: unit + ref: "classes/providers#TestOpenAIAdapterPayload, TestAnthropicAdapterPayload; classes/services#TestAIServiceFailures, TestAIServiceStream; classes/factories#TestPromptFactory" + status: pass + human_judgment: false + - id: D3 + description: "AI models table, admin list and form, settings lookups with the default-model quirk, one-time importer" + requirement: INTG-02 + verification: + - kind: unit + ref: "plugins/golem15/golem#TestImportSettings, TestAdminModelsForm; classes/services#TestDefaultModelQuirk" + status: pass + human_judgment: false + - id: D4 + description: "SSRF guard port and the trusted/guarded split" + requirement: INTG-02 + verification: + - kind: unit + ref: "classes/security#TestSSRFGuard; classes/services#TestAdminModelTrusted; modules/fetchguard#TestIsPrivateAddr, ExampleIsPrivateAddr" + status: pass + human_judgment: false + - id: D5 + description: "AI tier resolution (admin vision model trusted, user and org guarded, no fallback) and ai-credential/test" + requirement: INTG-02 + verification: + - kind: unit + ref: "plugins/golem15/fonoteka#TestAICredentialTestRoute, TestAdminVisionTier, TestResolveAIConfigPrecedence" + status: pass + - kind: parity + ref: "parity#TestParityCorpus/POST___fonoteka_api_v1_ai-credential_test_jwt (7 cases, 4 sidecars)" + status: pass + human_judgment: false + - id: D6 + description: "Album recognition on the JWT and token groups with PHP's answers" + requirement: INTG-02 + verification: + - kind: unit + ref: "plugins/golem15/fonoteka/classes#TestRecognizeAlbums; plugins/golem15/fonoteka#TestRecognizeRoutes" + status: pass + - kind: parity + ref: "parity#TestParityCorpus/POST___fonoteka_api_v1_albums_recognize_jwt (10 cases), POST__api_v1_fonoteka_albums_recognize_personal_token (3 cases); TestParityCorpus/coverage 168 ported, 3 pending" + status: pass + human_judgment: false + +duration: 69min +completed: 2026-10-03 +status: complete +--- + +# Phase 14 Plan 04: sm-golem-plugin and AI cover recognition Summary + +**A new shared plugin, sm-golem-plugin, brings the Golem AI layer to Go: SDK-free OpenAI-compatible and Anthropic adapters through the guarded client, the operator's AI models with an admin screen and a one-time importer, and the exact SSRF guard. On top of it a collector can test an AI key and photograph albums in the Nuxt app or through MCP, and gets PHP's answers, recorded from PHP and replayed offline.** + +## Performance + +- **Duration:** 69 min +- **Started:** 2026-10-03T20:27:58Z +- **Completed:** 2026-10-03T21:37:08Z +- **Tasks:** 4 +- **Commits:** 3 in sm-golem-plugin (pushed), 8 in fonoteka.go, 1 code commit in summercms.go (plus this summary) +- **Files:** 34 in the plugin (+4505), 105 in fonoteka.go (+2664/-176), 5 in summercms.go (+55) + +## Accomplishments + +- **sm-golem-plugin (D-01).** The empty remote was cloned at `plugins/golem15/golem`, built, committed and pushed (`master` at `c93e2d9`, in sync with origin), then registered with `git submodule add`. fonoteka.go loads `golem15.golem` before `golem15.fonoteka`: `go.work`, both `go.mod` files, `summer.yaml`, the regenerated `plugins.gen.go` and `app.PluginIDs` name it. `golem15.fonoteka` requires it, and every test activation list names it. +- **AI layer (D-02, D-04).** + - `OpenAIAdapter` and `AnthropicAdapter` are hand-written JSON mappings, with no SDK. The OpenAI adapter maps `max_completion_tokens`, `response_format` and `reasoning_effort` as PHP does. The Anthropic adapter sets the `max_tokens` chain, builds image and document blocks and maps usage. + - `AIService` covers every PHP call except faces and chat, with PHP's messages. + - Requests go through `fetchguard`: `TrustedMode` for operator models and `PublicOnlyMode` for everyone else. They carry PHP curl's `Accept: */*` and no User-Agent. +- **Models and admin (D-03, D-18).** `golem15_golem_models` keeps an encrypted, write-only `api_key` and has a cabana list and form under `golem15.golem.access_settings`. Its lookups keep PHP's default-model quirk. `golem:import-settings` copies the `golem_settings` repeater once and encrypts each key. +- **SSRF (D-05, D-20).** `security.AssertSafeURL` ports SSRFGuard: https only, the allowlist (overridable with `GOLEM15_SSRF_ALLOWED_HOSTS`), and the resolve-time check through the new `fetchguard.IsPrivateAddr`. User and org `base_url` overrides are checked when they are resolved. A refused URL answers Winter's 500 page. +- **AI tier (D-03).** `AIConfig.Trusted` is set only by the admin seam, which Boot fills from the golem vision model. A normal user never gets the admin model, and an org-locked member never falls back to it. +- **Routes (INTG-02, D-07).** + - `POST ai-credential/test` (JWT) tests an inline key or the caller's own stored key. + - `POST albums/recognize` runs on the JWT group and on the token group (`inv.scope:ai`). The checks run in PHP's order: gate, limiter (10 per 60 s), validation, image guard, then the AI tier. It answers 200 with the albums, 200 with `recognition_truncated`, or 502. + - `RecognizeAlbums` sends PHP's system prompt verbatim, with the collection's genre hint and the language directive. It retries once, then normalises the answer as PHP does. +- **Parity.** 20 cases were recorded against PHP through `summer parity:upstream`, 12 of them with sidecars, covering both the OpenAI and the Anthropic exchanges. The cases include the admin tier through the global vision model, truncation on `length` and on `max_tokens`, an empty answer, a provider error, a JPEG polyglot refused before any upstream call, and the unsafe and non-allowlisted base URLs (500 page, no upstream). The corpus has 168 routes ported and passing and 3 pending (the D-09 routes). `check_corpus --require-recorded --check-secrets` is green. A mutation of one word in the prompt failed both recognize routes, which shows the upstream body is asserted. + +## Task Commits + +sm-golem-plugin (`git@git.golem15.com:golem15/sm-golem-plugin.git`, master, pushed): +1. `cd3f0f6` feat: golem15.golem plugin with the OpenAI-compatible adapter and AIService.Send (Task 1) +2. `da9fcde` feat: security.AssertSafeURL ports the SSRFGuard (Task 2) +3. `c93e2d9` feat: AI models admin, settings import, Anthropic adapter and the rest of AIService (Task 2) + +fonoteka.go (not pushed): +1. `91b3f72` chore(14-04): mount sm-golem-plugin at plugins/golem15/golem (`.gitmodules` + gitlink only) +2. `bf90131` feat(14-04): the application loads golem15.golem before golem15.fonoteka (Task 1) +3. `cf18030` fix(14-04): check_corpus header scans stay on one line (Task 1 blocker) +4. `bbb641b` feat(14-04): a collector tests an AI key and gets PHP's answer through golem15.golem (Task 1) +5. `cce718a` test(14-04): fonoteka's migration, schema and admin menu checks ignore golem15.golem's own (Task 2) +6. `8ec5e34` chore(14-04): bump sm-golem-plugin (gitlink only, Task 2) +7. `72b6edc` feat(14-04): the AI tier is chosen as PHP chooses it and unsafe base URLs get PHP's 500 page (Task 3) +8. `eea0b16` feat(14-04): a collector photographs albums in the Nuxt app or through MCP and gets PHP's recognised album list (Task 4) + +summercms.go: +- `7c2c433` feat(14-04): fetchguard.IsPrivateAddr exposes the dial guard's address classification (README and docs updated; TestDocsTree and docs:build --check pass) + +Tracer gate: after Task 1 the `` was re-run end to end (vet, the named tests with `-race`, the corpus, check_corpus, submodule status) and passed. The expansion tasks then went ahead. + +## Deviations from Plan + +### Auto-fixed issues + +**1. [Rule 3 - Blocking] `fetchguard.IsPrivateAddr` exported (framework)** +- **Issue:** The plan's guard must use "fetchguard's classification", but `isReservedOrPrivate` was unexported, and a plugin cannot reach it. +- **Fix:** Exported `IsPrivateAddr`, with a test, an example, README and `docs/services/outbound-http.md`. +- **Commit:** 7c2c433 (summercms.go) + +**2. [Rule 1 - Bug] check_corpus flagged masked credential headers** +- **Issue:** Once its `{{secret:ai-key}}` placeholder is stripped, `Authorization: Bearer` ran on through `\s+` into the next sorted header (`Content-Type:`). The same would happen to an unquoted `X-Api-Key`. +- **Fix:** The patterns stay on one line (`[ \t]`). Two clean vectors were added to the test. +- **Commit:** cf18030 + +**3. [Rule 3 - Blocking] The parity mismatch probe used ai-credential/test** +- **Fix:** `assertPortedMismatch` now probes `GET oauth-identities`, a D-09 route that stays pending. +- **Commit:** bbb641b + +**4. [Rule 3 - Blocking] Tests that pinned the plugin list** +- **Issue:** `activateAppPlugins` asserted two plugins. Once golem has migrations, a menu and a table, the migration-status indexes, the history-table list, the schema snapshot and the admin navigation test would all change. +- **Fix:** The activation-order check came first (in bf90131). The other checks were then made bump-neutral in cce718a: `withoutGolem`, the history-table filter, an `allowedDiffs` entry and `fonotekaCodes`. As a result the pointer bump `8ec5e34` changes only the gitlink and stays green, as the acceptance criterion requires. + +**5. [Rule 1 - Own omission] The part-file audit (T-12-17) rejected `spoofed.jpg`** +- **Fix:** The audit allows the deliberate polyglot (it must not be an image). The fix was folded into eea0b16 before anything was pushed, so every commit is green. + +**6. [Rule 2 - Correctness] Parity recording environment** +- `php_parity.sh` exports `GOLEM15_SSRF_ALLOWED_HOSTS=` (empty), so PHP uses the plugin default whatever the checkout's `.env` says. +- `fonoteka_reset.php` forgets the `golem_settings` query cache before `Settings::set`. Otherwise Winter saves onto the deleted row (the first `admin-truncated` recording answered 403 because of this). + +### Plan statements that conflict with PHP (parity kept) + +**7.** `ai-credential/test` tests the caller's own stored credential (PHP `UserAiConfig::fromCredential`), not "the stored user or org credential through ResolveAIConfig". + +**8.** `GenerateImage` does not run `AssertSafeURL` on the returned URL, because PHP's `generateImage` does not either; its caller does. The README says so, and `TestAIServiceFailures` checks the returned URL through the guard. + +### Plan details refined + +**9.** `AdminVisionModel(ctx, db)` and `SetAdminVisionModel(func(ctx, db))` replace the db-less signature. The seam is stateless and serves every app in the process, which is the 14-02 ReleaseFetcher precedent. Trusted is set by the seam itself. + +**10.** The plan's `aiModelConfig` lives in `classes` as `(*AIConfig).ModelConfig()`, because the api package cannot import the plugin root. `golem_wiring.go` holds the vision-model source. + +**11.** `NewAIService(db func() *gorm.DB, cfg)` and `services.ForApp(app)` resolve the database per call. `ModelConfig` also carries `AcceptsImages`, `GeneratesImages` and `Position`, which the lookups and the default-model quirk need. `SendFileReader` uploads from a reader, which is the Go form of PHP's `sendFile(File)`. + +**12.** The genre hint is `GROUP BY name ORDER BY MAX(created_at) DESC LIMIT 20`, because Postgres rejects PHP's `DISTINCT` with `ORDER BY` on an unselected column. The recorded prompts match. + +**13.** No recorded 429 case for recognize. The Go replay keeps one app, and so one in-memory bucket, per route, so the JWT cases reach the limiter at most nine times. `TestRecognizeRoutes` asserts that the 11th call is 429. + +**14.** An admin model with an empty `base_url` uses the provider's default. PHP would build `"/messages"` and fail to connect. + +**15.** Extra tests beyond the plan: `TestAdminModelsForm` (write-only key through cabana's CRUD), `TestPromptFactory`, `TestIsPrivateAddr` and `ExampleIsPrivateAddr`. + +**16.** Commits land on `master` in both repos, as the sequential-executor instructions and the earlier Phase 14 plans do (`branching_strategy: none`). + +--- + +**Total deviations:** 16. Six are auto-fixes, two are plan conflicts resolved for parity, and eight are refinements. **Impact:** the only framework change is the small, documented `fetchguard.IsPrivateAddr`. There is no scope creep. + +## Issues Encountered + +- The shell aliases `rm` to its interactive form, and one merge step waited on its prompt. Later steps use `command rm -f`. +- A `pkill -f` whose pattern matched its own command line killed the shell. The PHP server was then stopped by PID. +- The recordings write nothing into the PHP checkout: the AI routes store no files, and `git status` there shows only the user's own pre-existing files. + +## Verification + +- sm-golem-plugin: `go vet ./...` and `go test ./...` pass, both standalone (`GOWORK=off`) and in the workspace. Every named test passes with `-race`. +- fonoteka.go: `go vet ./...` passes, and `go test ./... ./plugins/golem15/fonoteka/... ./plugins/golem15/golem/... -count=1` passes in every package. `go test ./... -short` passes. +- Parity: `TestParityCorpus` reports 171/171 recorded, 168 passing, 0 failing and 3 pending. `TestCheckCorpusPortedCaseStatus`, `TestUpstreamSidecarsAreReplayed` and `TestParityContract` pass. `check_corpus --require-recorded --check-secrets` passes, with 0 pending case-status mismatches. +- summercms.go: `go vet ./...` and `go test ./... -count=1` pass. `TestDocsTree` and `summer docs:build --check` pass. +- Acceptance greps for all four tasks pass. `expectedPortedRoutes` is 168, `phase14Absent` is gone, and `inv.scope:ai` sits on the recognize line. The pointer-bump commit touches only `plugins/golem15/golem`, and the plugin README names no consuming application. + +## Known Stubs + +None. + +## Threat Flags + +None. The new surfaces are in the plan's threat register: the AI models admin (T-14-24, permission `golem15.golem.access_settings`, write-only encrypted key), the user/org base URLs (T-14-22), trusted mode (T-14-23) and the recognize route (T-14-25..27). No key, token or DSN literal was pushed (T-14-29: the diff was scanned before each push; test keys are short fakes). + +## User Setup Required + +None for development. At cutover (Phase 15) the operator runs `golem:import-settings` once and confirms whether production needs `GOLEM15_SSRF_ALLOWED_HOSTS` (for example `.anthropic.com` for user Anthropic base URLs). This is research Open Question 1, still open. + +## Next Phase Readiness + +- 14-05 can follow the same submodule bootstrap for sm-feedback-plugin: empty clone at the mount path, push, `git submodule add`, a gitlink-only commit, and bump-neutral app tests first. +- 14-06 picks up the unit-test gate. INTG-02 is left Pending because 14-06 also declares it. + +--- +*Phase: 14-domain-jobs-and-external-integrations* +*Completed: 2026-10-03* + +## Self-Check: PASSED + +Every listed file exists; commits 7c2c433 (summercms.go), 91b3f72, bf90131, cf18030, bbb641b, cce718a, 8ec5e34, 72b6edc and eea0b16 (fonoteka.go) and cd3f0f6, da9fcde and c93e2d9 (sm-golem-plugin, on origin/master) are present.