golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0b25a1da247923f7a1c6863ae0a59cd56a51d4e6
summercms/.planning/phases/14-domain-jobs-and-external-integrations/14-04-SUMMARY.md

22 KiB
Raw Blame History

phase, plan, subsystem, tags, requires, provides, affects, actuals, plan_head_before, plan_head_after, app_repo_head_before, app_repo_head_after, plugin_repo_head_after, tech-stack, key-files, key-decisions, patterns-established, requirements-completed, coverage, duration, completed, status
phase plan subsystem tags requires provides affects actuals plan_head_before plan_head_after app_repo_head_before app_repo_head_after plugin_repo_head_after tech-stack key-files key-decisions patterns-established requirements-completed coverage duration completed status
14-domain-jobs-and-external-integrations 04 ai
sm-golem-plugin
golem15.golem
anthropic
openai
ai-service
ssrf
fetchguard
cabana
recognition
parity
upstream-sidecar
submodule
phase provides
14-domain-jobs-and-external-integrations 14-01 fetchguard.Client (TrustedMode, PublicOnlyMode, WithTransport), tide upstream sidecars and summer parity:upstream; 14-03 InboundLimits, phase14Routes and the per-case recording loop
phase provides
13-p-ytarium-api-wishlist-notifications-csv-credentials-public ResolveAIConfig, AIAllowed, the AdminVisionModel seam and the BYOK credential tables
New shared core plugin repo git.golem15.com/golem15/sm-golem-plugin (master pushed, cd3f0f6..c93e2d9), mounted in fonoteka.go at plugins/golem15/golem: plugin golem15.golem, package golem
golem AI layer: Prompt/Payload/AIResponse, OpenAI-compatible and Anthropic adapters over fetchguard (no SDK), AIService (Send, SendStream, SendFile, SendFilePath, SendFileReader, SendWithModel, SendToImageModel, SendToVisionModel, GenerateImage, Ask), PromptFactory
golem15_golem_models (encrypted write-only api_key) with its cabana list and form under golem15.golem.access_settings, the Settings lookups and golem:import-settings
security.AssertSafeURL / AllowedHosts / UnsafeURLError (SSRFGuard port, GOLEM15_SSRF_ALLOWED_HOSTS)
fetchguard.IsPrivateAddr (framework)
fonoteka: AIConfig.Trusted, SetAdminVisionModel/AdminVisionModel(ctx, db) fed by the golem vision model, UserAIConfig, guarded base_url overrides, RecognizeAlbums, AICredentialTest and AlbumRecognize on both groups
Parity: 20 PHP-recorded cases (7 ai-credential/test, 10 recognize JWT, 3 recognize token) with 12 upstream sidecars; ported 168, pending 3 (D-09)
14-05 sm-feedback-plugin (same submodule workflow)
14-06 unit tests and gate
15 cutover (GOLEM15_SSRF_ALLOWED_HOSTS
golem:import-settings)
tokens tasks commits app_repo_commits plugin_repo_commits
90000 4 1 8 3
f519261da6 7c2c43359f 28349433d77ba16c0dec7c87e41ef486996169f2 eea0b1607ef9b5d14b3a5186bf322e6eb45a4b9b c93e2d9
added patterns
A shared core plugin is born as an empty clone at its mount path, committed and pushed there, then registered with git submodule add; the application commits .gitmodules plus the gitlink alone, and later bumps as gitlink-only commits
Application tests that would change with a submodule bump are made bump-neutral first (filter the plugin's own history, tables and menu), so the bump commit stays a pure gitlink change and green
AI calls go through one AIService per app (services.ForApp); Trusted configs use fetchguard TrustedMode, every user or org config PublicOnlyMode after security.AssertSafeURL
PHP's raw-curl requests carry Accept: */* and no User-Agent; Go sends the same (an empty User-Agent value keeps net/http from adding its own)
Stateless app seams take the request's database (AdminVisionModel(ctx, db)), as 14-02's ReleaseFetcher does
created modified
../fonoteka.go/plugins/golem15/golem/ (whole repo
plugin.go, admin.go, README.md, config, lang, models, updates, controllers, console, classes/{valueobjects,providers,services,security,factories}, internal/pgtest)
../fonoteka.go/plugins/golem15/fonoteka/classes/album_recognition.go
../fonoteka.go/plugins/golem15/fonoteka/classes/album_recognition_test.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/recognize_controller.go
../fonoteka.go/plugins/golem15/fonoteka/golem_wiring.go
../fonoteka.go/plugins/golem15/fonoteka/ai_routes_test.go
../fonoteka.go/parity/upstream/scripts/{openai-chat-401,openai-chat-pong,anthropic-messages-pong,openai-recognize,openai-recognize-truncated,openai-recognize-garbage,anthropic-recognize,anthropic-recognize-truncated}.yaml
../fonoteka.go/parity/fixtures/routes/files/spoofed.jpg
modules/fetchguard/ip.go
modules/fetchguard/README.md
docs/services/outbound-http.md
../fonoteka.go/.gitmodules
../fonoteka.go/go.work
../fonoteka.go/go.mod
../fonoteka.go/summer.yaml
../fonoteka.go/plugins.gen.go
../fonoteka.go/app/app.go
../fonoteka.go/plugins/golem15/fonoteka/go.mod
../fonoteka.go/plugins/golem15/fonoteka/plugin.go
../fonoteka.go/plugins/golem15/fonoteka/routes.go
../fonoteka.go/plugins/golem15/fonoteka/classes/ai_config_resolver.go
../fonoteka.go/plugins/golem15/fonoteka/classes/gates.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/credentials_controller.go
../fonoteka.go/plugins/golem15/fonoteka/controllers/api/inbound_limits.go
../fonoteka.go/parity/{manifest.yaml,parity_test.go,parity_contract_test.go,fonoteka_seed_test.go,fonoteka_reset.php,php_parity.sh,check_corpus.go,README.md}
ai-credential/test tests the caller's own stored credential only (PHP UserAiConfig::fromCredential), not the org or admin tier the plan named
AdminVisionModel takes the request's database: AdminVisionModel(ctx, db) and SetAdminVisionModel(func(ctx, db)); Boot installs a stateless reader of the golem vision model, Trusted set by the seam
The api package maps an AIConfig to the golem ModelConfig through (*classes.AIConfig).ModelConfig(), because the api package cannot import the plugin root
fetchguard.IsPrivateAddr is exported so the SSRF guard judges resolved addresses with the dial guard's table
GenerateImage keeps PHP's contract and does not fetch or check the returned URL; callers run security.AssertSafeURL (documented, tested)
An admin model with an empty base_url uses the provider default (PHP would post to /messages and fail to connect)
The admin api_key field is write-only: an update that sends it empty or redacted keeps the stored key
php_parity.sh exports GOLEM15_SSRF_ALLOWED_HOSTS empty so PHP records with the plugin default allowlist, which the Go replay uses
INTG-02 is not marked complete: 14-06 also declares it (requirements.ready-ids gate)
golem-vision and ai are additive seed extras on both sides, like discogs
Recorded multipart cases copy their part files into the specs directory (the recorder refuses symlinks)
INTG-02
id description requirement verification human_judgment
D1 sm-golem-plugin exists, is pushed and mounted; golem15.golem loads before golem15.fonoteka INTG-02
kind ref status
unit plugins/golem15/golem#TestGolemPluginBoot; plugins/golem15/fonoteka#TestRouteTablePhase12/13/14 pass
kind ref status
other git -C plugins/golem15/golem status --porcelain --branch → '## master...origin/master'; grep checks of .gitmodules, go.mod, summer.yaml, plugins.gen.go pass
false
id description requirement verification human_judgment
D2 OpenAI-compatible and Anthropic adapters and the full AIService with PHP's failure messages INTG-02
kind ref status
unit classes/providers#TestOpenAIAdapterPayload, TestAnthropicAdapterPayload; classes/services#TestAIServiceFailures, TestAIServiceStream; classes/factories#TestPromptFactory pass
false
id description requirement verification human_judgment
D3 AI models table, admin list and form, settings lookups with the default-model quirk, one-time importer INTG-02
kind ref status
unit plugins/golem15/golem#TestImportSettings, TestAdminModelsForm; classes/services#TestDefaultModelQuirk pass
false
id description requirement verification human_judgment
D4 SSRF guard port and the trusted/guarded split INTG-02
kind ref status
unit classes/security#TestSSRFGuard; classes/services#TestAdminModelTrusted; modules/fetchguard#TestIsPrivateAddr, ExampleIsPrivateAddr pass
false
id description requirement verification human_judgment
D5 AI tier resolution (admin vision model trusted, user and org guarded, no fallback) and ai-credential/test INTG-02
kind ref status
unit plugins/golem15/fonoteka#TestAICredentialTestRoute, TestAdminVisionTier, TestResolveAIConfigPrecedence pass
kind ref status
parity parity#TestParityCorpus/POST___fonoteka_api_v1_ai-credential_test_jwt (7 cases, 4 sidecars) pass
false
id description requirement verification human_judgment
D6 Album recognition on the JWT and token groups with PHP's answers INTG-02
kind ref status
unit plugins/golem15/fonoteka/classes#TestRecognizeAlbums; plugins/golem15/fonoteka#TestRecognizeRoutes pass
kind ref status
parity parity#TestParityCorpus/POST___fonoteka_api_v1_albums_recognize_jwt (10 cases), POST__api_v1_fonoteka_albums_recognize_personal_token (3 cases); TestParityCorpus/coverage 168 ported, 3 pending pass
false
69min 2026-10-03 complete

Phase 14 Plan 04: sm-golem-plugin and AI cover recognition Summary

A new shared plugin, sm-golem-plugin, brings the Golem AI layer to Go: SDK-free OpenAI-compatible and Anthropic adapters through the guarded client, the operator's AI models with an admin screen and a one-time importer, and the exact SSRF guard. On top of it a collector can test an AI key and photograph albums in the Nuxt app or through MCP, and gets PHP's answers, recorded from PHP and replayed offline.

Performance

  • Duration: 69 min
  • Started: 2026-10-03T20:27:58Z
  • Completed: 2026-10-03T21:37:08Z
  • Tasks: 4
  • Commits: 3 in sm-golem-plugin (pushed), 8 in fonoteka.go, 1 code commit in summercms.go (plus this summary)
  • Files: 34 in the plugin (+4505), 105 in fonoteka.go (+2664/-176), 5 in summercms.go (+55)

Accomplishments

  • sm-golem-plugin (D-01). The empty remote was cloned at plugins/golem15/golem, built, committed and pushed (master at c93e2d9, in sync with origin), then registered with git submodule add. fonoteka.go loads golem15.golem before golem15.fonoteka: go.work, both go.mod files, summer.yaml, the regenerated plugins.gen.go and app.PluginIDs name it. golem15.fonoteka requires it, and every test activation list names it.
  • AI layer (D-02, D-04).
    • OpenAIAdapter and AnthropicAdapter are hand-written JSON mappings, with no SDK. The OpenAI adapter maps max_completion_tokens, response_format and reasoning_effort as PHP does. The Anthropic adapter sets the max_tokens chain, builds image and document blocks and maps usage.
    • AIService covers every PHP call except faces and chat, with PHP's messages.
    • Requests go through fetchguard: TrustedMode for operator models and PublicOnlyMode for everyone else. They carry PHP curl's Accept: */* and no User-Agent.
  • Models and admin (D-03, D-18). golem15_golem_models keeps an encrypted, write-only api_key and has a cabana list and form under golem15.golem.access_settings. Its lookups keep PHP's default-model quirk. golem:import-settings copies the golem_settings repeater once and encrypts each key.
  • SSRF (D-05, D-20). security.AssertSafeURL ports SSRFGuard: https only, the allowlist (overridable with GOLEM15_SSRF_ALLOWED_HOSTS), and the resolve-time check through the new fetchguard.IsPrivateAddr. User and org base_url overrides are checked when they are resolved. A refused URL answers Winter's 500 page.
  • AI tier (D-03). AIConfig.Trusted is set only by the admin seam, which Boot fills from the golem vision model. A normal user never gets the admin model, and an org-locked member never falls back to it.
  • Routes (INTG-02, D-07).
    • POST ai-credential/test (JWT) tests an inline key or the caller's own stored key.
    • POST albums/recognize runs on the JWT group and on the token group (inv.scope:ai). The checks run in PHP's order: gate, limiter (10 per 60 s), validation, image guard, then the AI tier. It answers 200 with the albums, 200 with recognition_truncated, or 502.
    • RecognizeAlbums sends PHP's system prompt verbatim, with the collection's genre hint and the language directive. It retries once, then normalises the answer as PHP does.
  • Parity. 20 cases were recorded against PHP through summer parity:upstream, 12 of them with sidecars, covering both the OpenAI and the Anthropic exchanges. The cases include the admin tier through the global vision model, truncation on length and on max_tokens, an empty answer, a provider error, a JPEG polyglot refused before any upstream call, and the unsafe and non-allowlisted base URLs (500 page, no upstream). The corpus has 168 routes ported and passing and 3 pending (the D-09 routes). check_corpus --require-recorded --check-secrets is green. A mutation of one word in the prompt failed both recognize routes, which shows the upstream body is asserted.

Task Commits

sm-golem-plugin (git@git.golem15.com:golem15/sm-golem-plugin.git, master, pushed):

  1. cd3f0f6 feat: golem15.golem plugin with the OpenAI-compatible adapter and AIService.Send (Task 1)
  2. da9fcde feat: security.AssertSafeURL ports the SSRFGuard (Task 2)
  3. c93e2d9 feat: AI models admin, settings import, Anthropic adapter and the rest of AIService (Task 2)

fonoteka.go (not pushed):

  1. 91b3f72 chore(14-04): mount sm-golem-plugin at plugins/golem15/golem (.gitmodules + gitlink only)
  2. bf90131 feat(14-04): the application loads golem15.golem before golem15.fonoteka (Task 1)
  3. cf18030 fix(14-04): check_corpus header scans stay on one line (Task 1 blocker)
  4. bbb641b feat(14-04): a collector tests an AI key and gets PHP's answer through golem15.golem (Task 1)
  5. cce718a test(14-04): fonoteka's migration, schema and admin menu checks ignore golem15.golem's own (Task 2)
  6. 8ec5e34 chore(14-04): bump sm-golem-plugin (gitlink only, Task 2)
  7. 72b6edc feat(14-04): the AI tier is chosen as PHP chooses it and unsafe base URLs get PHP's 500 page (Task 3)
  8. eea0b16 feat(14-04): a collector photographs albums in the Nuxt app or through MCP and gets PHP's recognised album list (Task 4)

summercms.go:

  • 7c2c433 feat(14-04): fetchguard.IsPrivateAddr exposes the dial guard's address classification (README and docs updated; TestDocsTree and docs:build --check pass)

Tracer gate: after Task 1 the <verify> was re-run end to end (vet, the named tests with -race, the corpus, check_corpus, submodule status) and passed. The expansion tasks then went ahead.

Deviations from Plan

Auto-fixed issues

1. [Rule 3 - Blocking] fetchguard.IsPrivateAddr exported (framework)

  • Issue: The plan's guard must use "fetchguard's classification", but isReservedOrPrivate was unexported, and a plugin cannot reach it.
  • Fix: Exported IsPrivateAddr, with a test, an example, README and docs/services/outbound-http.md.
  • Commit: 7c2c433 (summercms.go)

2. [Rule 1 - Bug] check_corpus flagged masked credential headers

  • Issue: Once its {{secret:ai-key}} placeholder is stripped, Authorization: Bearer ran on through \s+ into the next sorted header (Content-Type:). The same would happen to an unquoted X-Api-Key.
  • Fix: The patterns stay on one line ([ \t]). Two clean vectors were added to the test.
  • Commit: cf18030

3. [Rule 3 - Blocking] The parity mismatch probe used ai-credential/test

  • Fix: assertPortedMismatch now probes GET oauth-identities, a D-09 route that stays pending.
  • Commit: bbb641b

4. [Rule 3 - Blocking] Tests that pinned the plugin list

  • Issue: activateAppPlugins asserted two plugins. Once golem has migrations, a menu and a table, the migration-status indexes, the history-table list, the schema snapshot and the admin navigation test would all change.
  • Fix: The activation-order check came first (in bf90131). The other checks were then made bump-neutral in cce718a: withoutGolem, the history-table filter, an allowedDiffs entry and fonotekaCodes. As a result the pointer bump 8ec5e34 changes only the gitlink and stays green, as the acceptance criterion requires.

5. [Rule 1 - Own omission] The part-file audit (T-12-17) rejected spoofed.jpg

  • Fix: The audit allows the deliberate polyglot (it must not be an image). The fix was folded into eea0b16 before anything was pushed, so every commit is green.

6. [Rule 2 - Correctness] Parity recording environment

  • php_parity.sh exports GOLEM15_SSRF_ALLOWED_HOSTS= (empty), so PHP uses the plugin default whatever the checkout's .env says.
  • fonoteka_reset.php forgets the golem_settings query cache before Settings::set. Otherwise Winter saves onto the deleted row (the first admin-truncated recording answered 403 because of this).

Plan statements that conflict with PHP (parity kept)

7. ai-credential/test tests the caller's own stored credential (PHP UserAiConfig::fromCredential), not "the stored user or org credential through ResolveAIConfig".

8. GenerateImage does not run AssertSafeURL on the returned URL, because PHP's generateImage does not either; its caller does. The README says so, and TestAIServiceFailures checks the returned URL through the guard.

Plan details refined

9. AdminVisionModel(ctx, db) and SetAdminVisionModel(func(ctx, db)) replace the db-less signature. The seam is stateless and serves every app in the process, which is the 14-02 ReleaseFetcher precedent. Trusted is set by the seam itself.

10. The plan's aiModelConfig lives in classes as (*AIConfig).ModelConfig(), because the api package cannot import the plugin root. golem_wiring.go holds the vision-model source.

11. NewAIService(db func() *gorm.DB, cfg) and services.ForApp(app) resolve the database per call. ModelConfig also carries AcceptsImages, GeneratesImages and Position, which the lookups and the default-model quirk need. SendFileReader uploads from a reader, which is the Go form of PHP's sendFile(File).

12. The genre hint is GROUP BY name ORDER BY MAX(created_at) DESC LIMIT 20, because Postgres rejects PHP's DISTINCT with ORDER BY on an unselected column. The recorded prompts match.

13. No recorded 429 case for recognize. The Go replay keeps one app, and so one in-memory bucket, per route, so the JWT cases reach the limiter at most nine times. TestRecognizeRoutes asserts that the 11th call is 429.

14. An admin model with an empty base_url uses the provider's default. PHP would build "/messages" and fail to connect.

15. Extra tests beyond the plan: TestAdminModelsForm (write-only key through cabana's CRUD), TestPromptFactory, TestIsPrivateAddr and ExampleIsPrivateAddr.

16. Commits land on master in both repos, as the sequential-executor instructions and the earlier Phase 14 plans do (branching_strategy: none).

Total deviations: 16. Six are auto-fixes, two are plan conflicts resolved for parity, and eight are refinements. Impact: the only framework change is the small, documented fetchguard.IsPrivateAddr. There is no scope creep.

Issues Encountered

  • The shell aliases rm to its interactive form, and one merge step waited on its prompt. Later steps use command rm -f.
  • A pkill -f whose pattern matched its own command line killed the shell. The PHP server was then stopped by PID.
  • The recordings write nothing into the PHP checkout: the AI routes store no files, and git status there shows only the user's own pre-existing files.

Verification

  • sm-golem-plugin: go vet ./... and go test ./... pass, both standalone (GOWORK=off) and in the workspace. Every named test passes with -race.
  • fonoteka.go: go vet ./... passes, and go test ./... ./plugins/golem15/fonoteka/... ./plugins/golem15/golem/... -count=1 passes in every package. go test ./... -short passes.
  • Parity: TestParityCorpus reports 171/171 recorded, 168 passing, 0 failing and 3 pending. TestCheckCorpusPortedCaseStatus, TestUpstreamSidecarsAreReplayed and TestParityContract pass. check_corpus --require-recorded --check-secrets passes, with 0 pending case-status mismatches.
  • summercms.go: go vet ./... and go test ./... -count=1 pass. TestDocsTree and summer docs:build --check pass.
  • Acceptance greps for all four tasks pass. expectedPortedRoutes is 168, phase14Absent is gone, and inv.scope:ai sits on the recognize line. The pointer-bump commit touches only plugins/golem15/golem, and the plugin README names no consuming application.

Known Stubs

None.

Threat Flags

None. The new surfaces are in the plan's threat register: the AI models admin (T-14-24, permission golem15.golem.access_settings, write-only encrypted key), the user/org base URLs (T-14-22), trusted mode (T-14-23) and the recognize route (T-14-25..27). No key, token or DSN literal was pushed (T-14-29: the diff was scanned before each push; test keys are short fakes).

User Setup Required

None for development. At cutover (Phase 15) the operator runs golem:import-settings once and confirms whether production needs GOLEM15_SSRF_ALLOWED_HOSTS (for example .anthropic.com for user Anthropic base URLs). This is research Open Question 1, still open.

Next Phase Readiness

  • 14-05 can follow the same submodule bootstrap for sm-feedback-plugin: empty clone at the mount path, push, git submodule add, a gitlink-only commit, and bump-neutral app tests first.
  • 14-06 picks up the unit-test gate. INTG-02 is left Pending because 14-06 also declares it.

Phase: 14-domain-jobs-and-external-integrations Completed: 2026-10-03

Self-Check: PASSED

Every listed file exists; commits 7c2c433 (summercms.go), 91b3f72, bf90131, cf18030, bbb641b, cce718a, 8ec5e34, 72b6edc and eea0b16 (fonoteka.go) and cd3f0f6, da9fcde and c93e2d9 (sm-golem-plugin, on origin/master) are present.

Reference in New Issue View Git Blame Copy Permalink