24 KiB
24 KiB
phase, verified, status, score, covered_files, covered_digest, covered_files_note, behavior_unverified, overrides_applied, mvp_mode_note, human_verification
| phase | verified | status | score | covered_files | covered_digest | covered_files_note | behavior_unverified | overrides_applied | mvp_mode_note | human_verification | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 14-domain-jobs-and-external-integrations | 2026-10-04T01:11:09Z | human_needed | 6/6 roadmap success criteria verified; plan truths 54/56 verified, 2 deviate from their literal text to keep PHP behaviour (14-03 dry_run and second-apply covers, routed to human for an override decision) |
|
v2:sha256:63033691c194f6c817c36b8fe27cc414ecb1affd35099a92cdf9844275fd1b2e | verification.fingerprint covers only paths under the summercms.go root, so the fonoteka.go, sm-golem-plugin and sm-feedback-plugin implementation files (most of this phase) are not in the digest. Their state at verification: fonoteka.go cd393e4, sm-golem-plugin 6646d10, sm-feedback-plugin 3057719, all with clean trees. | 0 | 0 | ROADMAP marks Phase 14 mode: mvp, but the goal is not a User Story and no 14-*-PLAN.md carries one. As in Phases 1, 3, 5 and 8 to 13, the six ROADMAP success criteria are the contract, User Flow Coverage is derived from them, and plan must_haves are supporting evidence. |
|
Phase 14: Domain jobs and external integrations Verification Report
Phase Goal: The domain-specific River jobs (CSV import write, Discogs match, wishlist digest), the reindex command, the Discogs client and AI cover recognition are ported on top of the Phase 11 jobs/realtime/search infrastructure and the Phase 13 API surface they serve. Verified: 2026-10-04T01:11:09Z Status: human_needed Re-verification: No, initial verification
MVP note: ROADMAP marks this phase mode: mvp, but the goal is not a User Story. As in Phases 1, 3, 5 and 8 to 13, the six ROADMAP success criteria are the contract.
User Flow Coverage
| # | Step (from SC) | Expected | Evidence in codebase | Status |
|---|---|---|---|---|
| 1 | Collector uploads a CSV; match job runs | Rows matched against the collection and Discogs; a rate limit pauses and re-dispatches | csv_match_job.go deliverCsvMatch / pause; jobs.go:37 conga.Timeout(240 * time.Second); TestCsvMatchJob (php-recorded, re-dispatch delay, 240 s Describe) PASS |
✓ |
| 2 | Collector commits; import job writes albums | Canonical, draft or CSV rows written; one bulk event | csv_import_job.go; TestCsvImportJob, TestPhase14Jobs PASS |
✓ |
| 3 | Subscriber gets one digest per 30-minute window | Queue row deleted, one mail | wishlist_digest_job.go (DELETE-gated mail); Phase 13 EnqueueWishlistDigest dispatches only on insert with 1800 s delay; TestWishlistDigestJob PASS |
✓ |
| 4 | Operator runs fonoteka:reindex [--drop-old-items-index] |
Tenantless abort, rebuild, zero collection_id:=0 docs, legacy drop | console/reindex.go, matches ReindexAlbums.php line for line; TestReindexCommand PASS |
✓ |
| 5 | Collector matches, applies and imports Discogs releases; MCP fetches cover and price | PHP's bodies and limiter answers | 8 routes in routes.go; parity 172/172 passing |
✓ (see CR-01) |
| 6 | Collector or MCP photographs albums | Anthropic or OpenAI-compatible recognition; admin tier uses the global vision model | sm-golem-plugin adapters; golem_wiring.go → SetAdminVisionModel; recognize parity cases (20) PASS |
✓ |
| 7 | Visitor submits feedback; collector hides widget | 202 submit, config, me/hidden, feedback_widget_hidden on user payload |
sm-feedback-plugin routes and listener; 28 parity cases PASS | ✓ |
Goal Achievement
Observable Truths (ROADMAP Success Criteria)
| # | Truth | Status | Evidence |
|---|---|---|---|
| 1 | The CSV import write job and the self-redispatching Discogs match job (240 s timeout, re-dispatching with a delay on a Discogs rate-limit error) both complete correctly | ✓ VERIFIED | Workers registered in jobs.go:45-49 on queues that conga serves (knownQueues adds every registered job's queue). pause() dispatches a new match job with Delay = max(RetryAfter, SecondsUntilAvailable) under a row lock and completes the old one {"paused":"discogs_rate_limited"}. Behavioural tests run by verifier: TestCsvMatchJob (subtests "a rate limit beyond the budget re-dispatches with the computed delay", "sub-second remainder delays one second", "registered with a 240 s timeout", "php-recorded"), TestCsvImportJob, TestPhase14Jobs, TestCsvWR02: all PASS. Cancel-branch write is unconditional, as in PHP AlbumCsvMatchJob.php:100-103 (WR-01, warning). |
| 2 | The wishlist digest job coalesces a 30-minute window and deletes its queue row on completion | ✓ VERIFIED | EnqueueWishlistDigest upserts the queue row and dispatches only when the insert created it (RETURNING (xmax = 0)), delay WishlistDigestDelay = 1800 * time.Second. deliverWishlistDigest deletes the row and mails only if the DELETE removed it. TestWishlistDigestJob asserts row count 0 after send, skip on missing or zero row, PL/EN templates and a single mail for two runs: PASS. |
| 3 | The reindex command asserts zero collection_id-0 documents before and after and can drop the legacy index |
✓ VERIFIED | console/reindex.go: aborts on collection_id IS NULL OR <= 0, Flush, EnsureIndex, upsert by id in batches of 500, collection_id:=0 search must be 0, --drop-old-items-index drops only prefix+golem15_fonoteka_items via beachcomber.DropIndex with PHP's two messages. Registered in plugin.go:296. TestReindexCommand (batch ordering, empty table, integrity fail, tenantless abort, legacy drop twice) PASS. |
| 4 | Discogs client enforces proactive rate threshold, bounded wait budget, retry-after fallback and host-locked cover fetch; the 8 Discogs routes and the CSV row-edit selected_discogs_id pick pass the parity diff |
✓ VERIFIED | classes/discogs/client.go: code-constant https://api.discogs.com, fetchguard AllowHostsMode, Acquire + 429 loop with RegisterRetryAfter fallback and deadline budget. rate_store.go UNLOGGED table plus one INSERT … ON CONFLICT … WHERE … RETURNING. discogs_wiring.go installs the real ReleaseFetcher (plugin.go:138). Routes mounted at routes.go:102-112, 192-194, 224, 298. Verifier ran TestParityCorpus: recorded 175/175 passing 172 failing 0 unrecorded 0 pending 3, and TestUpstreamSidecarsAreReplayed PASS. TestDiscogsClientStatuses, TestRateLimiterBudget, TestRegisterRetryAfter, TestDiscogsRateWindowConcurrent, TestCoverFetcherHostLock, TestPHPTruthDiscogs PASS. CR-01 (lost update under concurrency) does not falsify this criterion; it is escalated to human. |
| 5 | AI cover recognition through Anthropic and OpenAI-compatible adapters over the guarded client with per-credential overrides; albums/recognize (JWT and token groups) and ai-credential/test pass parity; admin tier uses the backend global vision model |
✓ VERIFIED | sm-golem-plugin classes/providers/{anthropic,openai}.go are hand-rolled (x-api-key, anthropic-version, /messages; Bearer, /chat/completions, max_completion_tokens, reasoning_effort). No vendor SDK in the module graph (gate). AIService.client uses TrustedMode for admin models and PublicOnlyMode otherwise. golem_wiring.go sets SetAdminVisionModel(golemVisionModel) with Trusted: true, called at plugin.go:141. Routes at routes.go:109, 218, 285 (inv.scope:ai). TestRecognizeRoutes, TestAICredentialTestRoute PASS. Parity cases are in the 172 passing. sm-golem-plugin go test ./... PASS. |
| 6 | Feedback submissions, widget config and hide preference work (sitemap dropped, D-14); oauth-client, prune-notifications and reindex commands run correctly; oauth-identities and token me stay pending (D-09) |
✓ VERIFIED | sm-feedback-plugin routes.go: config and submit with throttle buckets, PUT me/hidden, embed.js. embed.js sha256 is identical to PHP's (a76d27dc…). The GetApiArrayEvent listener adds feedback_widget_hidden. sm-feedback-plugin go test ./... PASS. Its 28 parity cases are in the 172 passing. Commands() registers OAuthClient, PruneNotifications and Reindex. The schedule entry fonoteka:prune-notifications Daily resolves (TestFonotekaSchedulePrune PASS). TestPruneNotifications (90-day boundary, two concurrent runs delete once) PASS. Exactly 3 pending manifest entries, all D-09 routes; Phase 14.1 owns them. |
Score: 6/6 roadmap truths verified (0 present, behavior-unverified)
Plan must_haves (supporting evidence)
All 56 plan truths were checked. 54 hold, including both 14-02 backstop truths: reindex ordering is pinned by the TestReindexCommand subtest "every live album by id in batches of 500", and prune concurrency by TestPruneNotifications "two concurrent runs". Two 14-03 statements deviate from their literal text to keep PHP behaviour:
| Plan | Statement | Actual | Disposition |
|---|---|---|---|
| 14-03 truth 4 / prohibition | "dry_run writes nothing" |
Writes covers and catalog fields and stages only DRAFT_FIELDS, exactly as AlbumReleaseApplicator.php:75-133 |
Human: suggested override |
| 14-03 backstop | "second identical apply imports no second cover" | Re-imports covers, as PHP's recorded twice case does |
Human: suggested override |
The intermediate counts in plan truths (14-03 "165 ported, 6 pending", 14-04 "168") were superseded by 14-05 and 14-06. The final state, 175/172/3, is the one that holds.
Required Artifacts
| Artifact | Status | Details |
|---|---|---|
summercms.go modules/fetchguard/client.go, modules/tide/upstream.go, upstream_proxy.go, modules/sunscreen/sunscreen.go, modules/beachcomber/searchable.go, cmd/summer/parity.go |
✓ VERIFIED | Present, substantive, wired (internal/build/build.go emits sunscreen.InstallDefault; parity replay uses tide.NewUpstreamFake through fetchguard.WithTransport) |
fonoteka.go classes/discogs/{client,rate_store,rate_limiter,cover_fetcher}.go, updates/22_discogs_rate_windows_table.go, csv_match_job.go, csv_import_job.go, wishlist_digest_job.go, console/{reindex,prune_notifications}.go, controllers/api/{release_match,inbound_limits,…}.go, classes/album_recognition.go, golem_wiring.go, discogs_wiring.go |
✓ VERIFIED | All present (32–536 lines), registered in plugin.go/jobs.go/routes.go |
sm-golem-plugin plugin.go, classes/services/ai_service.go, classes/security/ssrf_guard.go, models/ai_model.go |
✓ VERIFIED | Submodule at 6646d10, listed in summer.yaml before fonoteka |
sm-feedback-plugin plugin.go, controllers/api/feedback_api_controller.go, classes/g15office_client.go, assets/js/embed.js |
✓ VERIFIED | Submodule at 3057719, listed in summer.yaml |
scripts/check-phase14.sh, parity/discogs_truth_tables.php, phase14_security_test.go |
✓ VERIFIED | --evidence stage run by verifier: passed |
Key Link Verification
| From | To | Via | Status |
|---|---|---|---|
plugin.go |
classes.SetReleaseFetcher |
discogsReleaseFetcher{} at Boot (line 138) |
WIRED |
plugin.go |
classes.SetAdminVisionModel |
installGolemWiring() (line 141) |
WIRED |
discogs/client.go |
fetchguard | fetchguard.NewClient(AllowHostsMode, api.discogs.com) |
WIRED |
jobs.go |
match/import/digest workers | conga.Job(...) with OnQueue, 240 s timeout |
WIRED |
routes.go |
11 Phase 14 handlers | JWT and token groups with scopes and throttles | WIRED (TestRouteTablePhase14 PASS) |
feedback plugin.go |
user payload | GetApiArrayEvent listener |
WIRED |
parity_test.go |
tide sidecars | LoadUpstream + WithTransport |
WIRED (TestUpstreamSidecarsAreReplayed PASS) |
Data-Flow Trace (Level 4)
| Artifact | Data | Source | Real data | Status |
|---|---|---|---|---|
| Discogs routes | release, search, price | discogs.ForUser → guarded GET → csv.DecodeOrdered |
Yes (replayed from recorded sidecars) | ✓ FLOWING |
| Recognize | album list | AIService → adapter → RecognizeAlbums normaliser |
Yes | ✓ FLOWING |
| Admin AI tier | vision model | services.VisionModel(ctx, db) from golem15_golem_models |
Yes | ✓ FLOWING |
| Digest mail | itemCount, ownerName, wishlistName | queue row, collections, users | Yes | ✓ FLOWING |
Behavioral Spot-Checks (run by verifier)
| Behavior | Command | Result | Status |
|---|---|---|---|
| Parity corpus | go test ./parity -run '^(TestParityCorpus|TestUpstreamSidecarsAreReplayed)$' |
recorded 175/175 passing 172 failing 0 pending 3 | ✓ PASS |
| Jobs, routes, threats, edges | go test ./plugins/golem15/fonoteka -run '^(TestCsvMatchJob|TestCsvImportJob|TestWishlistDigestJob|TestPhase14Jobs|TestCsvWR02|TestPhase14Edges|TestRouteTablePhase14|TestApplyReleaseModes|TestCoverPriceRoute|TestDiscogsCredentialTestRoute|TestAICredentialTestRoute|TestRecognizeRoutes|TestPhase14Threats|TestFonotekaSchedulePrune|TestWishlistDigestWorkerRegistered)$' |
15/15 PASS | ✓ PASS |
| Commands | go test ./plugins/golem15/fonoteka/console -run '^(TestPruneNotifications|TestReindexCommand)$' |
PASS | ✓ PASS |
| Discogs core | go test ./plugins/golem15/fonoteka/classes/discogs -run '^(TestPHPTruthDiscogs|TestDiscogsClientStatuses|TestRateLimiterBudget|TestRegisterRetryAfter|TestDiscogsRateWindowConcurrent|TestCoverFetcherHostLock|TestBucketID)$' |
7/7 PASS | ✓ PASS |
| Shared plugins | go test ./plugins/golem15/golem/... ./plugins/golem15/feedback/... |
all ok | ✓ PASS |
| Gate evidence | bash scripts/check-phase14.sh --evidence |
phase14 evidence passed | ✓ PASS |
--removal (about 25 min) was not re-run by the verifier. Its 43/43 result is taken from the 14-06 SUMMARY, and the named threat tests it depends on (TestPhase14Threats) pass.
Probe Execution
No scripts/*/tests/probe-*.sh exist and no plan declares a probe. The phase's runnable gate is scripts/check-phase14.sh. The orchestrator ran --go (build, vet, go test ./... green), and the verifier ran --evidence above.
Requirements Coverage
| Requirement | Source Plan | Status | Evidence |
|---|---|---|---|
| JOBS-02 | 14-02, 14-06 | ✓ SATISFIED | SC1 |
| JOBS-03 | 14-02, 14-06 | ✓ SATISFIED | SC2 |
| SRCH-02 | 14-01, 14-02, 14-06 | ✓ SATISFIED | SC3 |
| INTG-01 | 14-01, 14-02, 14-03, 14-06 | ✓ SATISFIED | SC4 (CR-01 escalated) |
| INTG-02 | 14-01, 14-04, 14-06 | ✓ SATISFIED | SC5 |
| API-08 | 14-01, 14-05, 14-06 | ✓ SATISFIED | SC6 |
| CLI-05 | 14-02, 14-06 | ✓ SATISFIED | SC6 (oauth-client from Phase 8, prune and reindex here) |
No orphaned requirements: REQUIREMENTS.md maps exactly these seven IDs to Phase 14, and all are marked Complete.
Anti-Patterns Found
No TBD, FIXME, XXX, TODO, HACK or placeholder markers in the Phase 14 implementation files of the four repos.
| File | Line | Pattern | Severity | Impact |
|---|---|---|---|---|
fonoteka/classes/album_write_service.go |
269 | GORM Save (all columns) after long Discogs I/O (CR-01) |
⚠️ Warning (escalated) | Lost update under concurrency, which PHP does not have |
fonoteka/csv_match_job.go |
127 | Unconditional cancel write; comment claims every status write is conditional (WR-01) | ⚠️ Warning | PHP parity; comment and D-10 narrative overstate the fix |
feedback/classes/submission_store.go |
85 | Client-chosen extension in public storage (WR-02) | ⚠️ Warning | Possible stored XSS surface |
fonoteka/routes.go |
218, 224 | Credential test routes unthrottled (WR-03) | ⚠️ Warning | PHP parity |
golem/console/import_settings.go |
152 | Map-keyed repeater read by 0..len-1 only (WR-05) | ⚠️ Warning | One-time import can drop models silently |
feedback/classes/sync_g15office.go |
174 | Non-idempotent on retry (WR-06) | ⚠️ Warning | Duplicate G15Office tasks |
Human Verification Required
1. CR-01 decision (recommended: fix before Phase 15)
Test: Run apply-release or MCP cover-price on an album while a concurrent PUT albums/{id} changes, for example, its notes.
Expected: The edit survives, as in PHP.
Why human: Parity cannot express concurrency, and no SC is literally falsified. Deciding whether this blocks the phase is a product call. The verifier recommends a column-scoped save that keeps the model rules and the broadcast hooks.
2. Override decision for the two 14-03 PHP-parity deviations
Test: Read the dry_run and second-apply rows above. Expected: Accept by adding the overrides below, or reject and re-plan. Why human: An agent instruction cannot accept a violated must-have.
3. Live vendor smoke test
Test: Use real Discogs, OpenAI and Anthropic credentials against the Go backend. Expected: Same results as PHP. Why human: Every upstream response in the corpus is scripted. Go's requests are proven equal to PHP's, but no real vendor response has ever been parsed.
4. Triage WR-01 to WR-06
Expected: Each one gets a disposition. Resolve WR-02 and WR-05 before cutover.
5. Admin screens and embed widget in a browser
Expected: Screens render and save; the widget loads, submits and hides.
Gaps Summary
No success criterion failed. The phase goal is achieved as the roadmap states it:
- all four job and command pieces run;
- the Discogs client enforces its rate rules;
- the 11 Phase 14 routes and 4 feedback routes pass the PHP parity diff, independently re-run: 175/175 recorded, 172 passing, 3 D-09 pending owned by Phase 14.1;
- recognition works through both adapters, with the admin vision tier wired.
The status is human_needed, not passed, for three reasons:
- CR-01 is real (confirmed in
applicator.go:105andcover_fetcher.go:345,378→album_write_service.go:269). It is a lost update that PHP does not have, in exactly the routes SC4 names. It does not falsify the literal criterion, which is parity diffs that cannot see concurrency, so it is escalated rather than marked FAILED. The verifier recommends fixing it in a quick task before Phase 15. - The two 14-03 statements were deliberately not met because PHP behaves otherwise. That is correct under the CLAUDE.md parity rule, but it needs a human-accepted override. Suggested entries:
overrides:
- must_have: "apply-release dry_run MUST NOT write anything"
reason: "PHP AlbumReleaseApplicator stages only DRAFT_FIELDS in dry run and writes covers and catalog fields; the recorded dry-run case is the contract. Go never writes a staged field (T-14-19)."
accepted_by: "{name}"
accepted_at: "{ISO timestamp}"
- must_have: "Edge (INTG-01 idempotency): applying the same release to the same album twice fills nothing on the second call and imports no second cover"
reason: "PHP re-imports covers on a second apply (recorded 'twice' case); Go matches it and fills no field twice."
accepted_by: "{name}"
accepted_at: "{ISO timestamp}"
- The vendor interop is proven only against scripted upstream responses. There are also six open review warnings, two of which (WR-02, WR-05) should be resolved before cutover.
Deferred items carried from deferred-items.md (informational): the Phase 12 cover-import replay still dials live image hosts, the one-off TestCoverFetcherHostLock failure under full -race load (it passed in the verifier's run), and the testcontainers start wait.
Verified: 2026-10-04T01:11:09Z Verifier: Claude (gsd-verifier)