diff --git a/.planning/debug/postgres-gate-field-types-conformance.md b/.planning/debug/postgres-gate-field-types-conformance.md new file mode 100644 index 0000000..62ae933 --- /dev/null +++ b/.planning/debug/postgres-gate-field-types-conformance.md @@ -0,0 +1,152 @@ +--- +status: diagnosed +trigger: "UAT gap G-16.2-1: bash scripts/check-phase16.2.sh postgres fails in ./modules/cabana: TestPhase10OpenAPIConformance: openapi_conformance_test.go:418: admin route GET /field-types has no conformance case" +created: 2026-10-10T17:10:00+02:00 +updated: 2026-10-10T17:28:00+02:00 +goal: find_root_cause_only +--- + +## Current Focus + +hypothesis: CONFIRMED — the postgres stage is red for two code defects, both written in 16.2 and both hidden because the checks only run in Docker-only test binaries that never ran before UAT: (1) two 16.2 admin routes have no conformCase in TestPhase10OpenAPIConformance; (2) sm-journal-plugin TestJournalSVGEmbed still asserts post content `type: mlmarkdown` after 16.2-03 changed it to `mlblogeditor`. +test: done — static diff, 3x direct repro, full cabana and Journal postgres-stage runs on this Docker host +expecting: n/a +next_action: return ROOT CAUSE FOUND (goal: find_root_cause_only); plan-phase --gaps owns the fix +bug_class: Bohrbug (deterministic; Go map iteration only changes WHICH of the two missing keys line 418 reports first) +reasoning_checkpoint: + hypothesis: "TestPhase10OpenAPIConformance fails because cabana.AdminAPIRoutes() (phase09Routes non-spa keys) has 56 entries and the conformCase list 54: GET /field-types and POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload, both added to the inventory in 16.2-01, have no case; once fixed, the same stage fails next on TestJournalSVGEmbed because it asserts `type: mlmarkdown` in models/post/fields.yaml, which 16.2-03 replaced with `type: mlblogeditor`." + confirming_evidence: + - "static diff of security_coverage_test.go phase09Routes vs conformCase keys: uncovered = [GET /field-types, POST .../fields/{field}/upload]; not-inventoried = []" + - "go test -count=3 -run '^TestPhase10OpenAPIConformance$' ./modules/cabana: FAIL 3/3, line 418 reports upload, field-types, upload" + - "Journal `go test -json -count=1 ./...` on this Docker host: only failure is TestJournalSVGEmbed, plugin_test.go:123 'post fields.yaml content is not mlmarkdown'; fields.yaml line 19 is `type: mlblogeditor` since fbd1c85" + - "cabana with -skip TestPhase10OpenAPIConformance: package PASS, 291 top-level tests, TestFieldTypeSavePostgres and TestListRelationSearchPostgres PASS with real Postgres queries in the output" + falsification_test: "If adding conformCases for exactly those two keys (and fixing the mlmarkdown assertion) still left the postgres stage red, or if the inventory/case diff showed a third key, the hypothesis would be incomplete." + fix_rationale: "Adding the two cases satisfies the exhaustiveness invariant and checks the real responses against admin.json (FieldTypesResponse, Envelope-cabana_FieldUploadResponse); updating the stale Journal assertion aligns it with the intended 16.2 field type. Separating the static inventory check from the Docker setup closes the masking path." + blind_spots: "The conformance cases themselves have not been written, so it is unverified that the fixture-backed responses decode cleanly with DisallowUnknownFields (e.g. FieldTypesResponse meta is `{}` from WriteData, which decodes into SuccessMeta; the upload case needs a conform plugin field type with Upload). Admin/editor/layer/docs stages were not re-run in this session." + candidate_causes: + - "code: missing conformCase entries for the two 16.2 routes (CONFIRMED)" + - "code: stale TestJournalSVGEmbed assertion vs the intended mlblogeditor change (CONFIRMED)" + - "environment: no Docker on the 16.2 execution host, so every testcontainers binary failed on docker.sock and was waved through as 'Docker-only' (CONFIRMED contributing)" + - "config/process: gate go stage FRAMEWORK_RUN omits TestPhase10OpenAPIConformance; the static inventory check runs after newConformEnv -> adminGorm; 16.2-01 PLAN/PATTERNS name phase09Routes + admin_openapi.go but not openapi_conformance_test.go (CONFIRMED contributing)" + - "data/environment: container start failure or flaky Postgres (ELIMINATED — containers start, 291 cabana + 66 Journal top-level tests pass)" + and_gate: "yes — the UAT failure needed the code defects AND the masking (static checks reachable only through testcontainers, no Docker during execution, go stage regex omits the test). Either code defect alone turns the stage red; the masking is why neither was caught before UAT." + +## Symptoms + +expected: `bash scripts/check-phase16.2.sh postgres` passes on a Docker host (cabana testcontainers suites incl. TestFieldTypeSavePostgres; Journal admin harness in ../sm-journal-plugin incl. TestPostsAdminResaveKeepsHTML, TestPostsAdminSaveSanitizesHTML, TestCategoriesAdminContent, TestCategoriesAdminBlogEditor) +actual: run 2026-10-10 16:57 CEST on a Docker host, the stage fails in ./modules/cabana: `--- FAIL: TestPhase10OpenAPIConformance` — `openapi_conformance_test.go:418: admin route GET /field-types has no conformance case`. The script stops there; the Journal Postgres suites never ran. +errors: openapi_conformance_test.go:418: admin route GET /field-types has no conformance case +reproduction: bash scripts/check-phase16.2.sh postgres (UAT Test 1) +started: Phase 16.2 (GET /field-types added in 16.2; discovered during UAT) + +## Eliminated + +- hypothesis: the postgres stage fails because of Docker/testcontainers environment problems (container start, timeouts) + evidence: in the UAT log the cabana package ran 90s and only TestPhase10OpenAPIConformance failed; in this session cabana (skip conformance) passed 291 top-level tests in 77s and the Journal run passed every package but root, with real SQL in the logs + timestamp: 2026-10-10T17:21:00+02:00 + +- hypothesis: only GET /field-types is missing a conformance case + evidence: the inventory/case diff shows POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload is uncovered too; `go test -count=3` reported the upload key in runs 1 and 3 and field-types in run 2 + timestamp: 2026-10-10T17:24:00+02:00 + +- hypothesis: the gap in the Journal suites is limited to the cabana abort (they would pass once cabana is green) + evidence: running the Journal stage command directly fails on TestJournalSVGEmbed (stale mlmarkdown assertion), which is in PG_JOURNAL_REQUIRE + timestamp: 2026-10-10T17:20:00+02:00 + +## Evidence + +- timestamp: 2026-10-10T17:12:00+02:00 + checked: .planning/debug/knowledge-base.md (Phase 0) + found: no knowledge base file exists; no prior pattern to test + implication: open-ended investigation + +- timestamp: 2026-10-10T17:13:00+02:00 + checked: scratchpad/pg.log (UAT run of the postgres stage) + found: detect_go reports failed only "modules/cabana TestPhase10OpenAPIConformance" plus the package; openapi_conformance_test.go:418 "admin route GET /field-types has no conformance case"; package took 90s (containers started fine) + implication: the only failing cabana test in that run was the conformance test; run_postgres aborts (set -e) before the Journal go_json call + +- timestamp: 2026-10-10T17:15:00+02:00 + checked: modules/cabana/openapi_conformance_test.go:129-450 + found: TestPhase10OpenAPIConformance calls newConformEnv(t) FIRST (line 130), then builds the conformCase list, then requires covered == inventory where inventory = cabana.AdminAPIRoutes() (export_test.go: every non-spa phase09Routes key). Missing key -> t.Fatalf at line 418 inside a `for key := range inventory` map loop (random order, so only the first missing key found is reported). + implication: the exhaustiveness check is purely static but cannot run without a Postgres container + +- timestamp: 2026-10-10T17:16:00+02:00 + checked: modules/cabana/auth_test.go:321-365 adminGorm + found: testing.Short() -> t.Skip; otherwise testcontainers postgres:16-alpine; a container error -> t.Fatalf("postgres: %v") + implication: on a host without Docker the conformance test fails at setup with a docker.sock error, masking the inventory check + +- timestamp: 2026-10-10T17:17:00+02:00 + checked: grep of conformCase keys vs modules/cabana/security_coverage_test.go phase09Routes + found: TWO inventoried non-spa routes have no conformCase: "GET /field-types" (inventory line 46) and "POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload" (inventory line 92). `grep 'fields/{field}/upload\|/field-types' openapi_conformance_test.go` -> no match. + implication: fixing only GET /field-types would leave the test failing on the upload route next + +- timestamp: 2026-10-10T17:18:00+02:00 + checked: git log -S on security_coverage_test.go and git log of openapi_conformance_test.go + found: GET /field-types entered the inventory in 717d46e (feat(16.2-01) compile, decode and load plugin field types), the upload route in 8fa92db (feat(16.2-01) ... uploads). openapi_conformance_test.go was last touched by 64d21c8/143eca5 (My account), b492e79 (markdown preview) — earlier route additions did add a conformCase in the same change; 16.2 did not. + implication: a convention ("new admin route => conformCase") that held before 16.2 was missed in 16.2-01 + +- timestamp: 2026-10-10T17:19:00+02:00 + checked: .planning/phases/16.2-*/ SUMMARY and VERIFICATION docs + found: no 16.2 plan mentions TestPhase10OpenAPIConformance; 16.2-01/03/05/06 SUMMARYs record "No Docker on this host (docker.sock permission denied)" and "the only failures ... are the testcontainers suites failing on docker.sock" — cabana is in that list. VERIFICATION marks the postgres stage "? SKIP (human item 1)". + implication: the conformance failure was hidden inside the blanket "Docker-only failure" bucket; nobody ran cabana's testcontainers suites before UAT + +- timestamp: 2026-10-10T17:20:00+02:00 + checked: scripts/check-phase16.2.sh go stage + found: FRAMEWORK_RUN regex lists TestPhase09ContractInventory and TestPhase09PermissionMatrix (static, both pass: they compare the mount with phase09Routes and admin.json paths) but NOT TestPhase10OpenAPIConformance; the go stage runs with -run "$FRAMEWORK_RUN" so the conformance test never runs there. The postgres stage runs the whole ./modules/cabana package and is the only stage that reaches it. + implication: no non-Docker gate covers the inventory-vs-conformance-case invariant + +- timestamp: 2026-10-10T17:21:00+02:00 + checked: admin/openapi/admin.json for both routes; modules/cabana/admin_openapi.go:264-276; field_type.go:834-871; contracts.go:245 + found: GET /field-types 200 documents $ref cabana.FieldTypesResponse ({data: []cabana.FieldTypeAssets, meta: cabana.SuccessMeta}, all of name/element/translatable/scripts/styles required); the handler writes WriteData(200, []FieldTypeAssets, nil) => {"data":[...],"meta":{}}. POST .../fields/{field}/upload 201 documents cabana.Envelope-cabana_FieldUploadResponse ({data:{url}, meta}); multipart part named "file". conformPlugin (openapi_conformance_test.go:818-843) does NOT implement pact.HasFormFieldTypes and the gadget fields.yaml has no plugin-typed field. + implication: a GET /field-types case can decode into cabana.FieldTypesResponse with ref "cabana.FieldTypesResponse" today (empty list); a meaningful case and any upload case need the conform fixture to register a plugin field type (with Upload, scripts) and a gadget field of that type + +- timestamp: 2026-10-10T17:19:00+02:00 + checked: route registrations added since 16.1/16.2 started (git diff 1ef21b0~1..HEAD of non-test modules/**/*.go in summercms.go; c60abc6..HEAD of non-test *.go in sm-journal-plugin) + found: summercms.go adds exactly two routes: g.Get("/field-types") and g.Post("/{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload"). sm-journal-plugin adds none (editor uploads go through the cabana route). + implication: the conformance gap is exactly those two routes; no other 16.2 route is missing a case + +- timestamp: 2026-10-10T17:19:30+02:00 + checked: ran the Journal postgres-stage command exactly as run_postgres does (`go test -json -count=1 -timeout 20m ./...` in ../sm-journal-plugin, which go.work/replace points at ../summercms.go) on this Docker host + found: packages classes, controllers, controllers/api, models, updates PASS; root package FAIL. Required tests: TestPostsFormCompiles PASS, TestPostsAdminCreateSmoke PASS, TestPostsAdminResaveKeepsHTML PASS, TestPostsAdminSaveSanitizesHTML PASS, TestCategoriesAdminContent PASS, TestCategoriesAdminBlogEditor PASS, TestJournalSVGEmbed FAIL — plugin_test.go:123 "post fields.yaml content is not mlmarkdown". 66 top-level tests passed, no skips, no "no tests to run". + implication: a SECOND, independent failure is queued behind the cabana one; fixing only the conformance case still leaves the postgres stage red + +- timestamp: 2026-10-10T17:20:00+02:00 + checked: sm-journal-plugin plugin_test.go:118-124, models/post/fields.yaml, git show fbd1c85 / 47f3032 + found: TestJournalSVGEmbed asserts strings.Contains(fields.yaml, "type: mlmarkdown") (a Phase 15 assertion). 16.2-03 commit fbd1c85 "feat(admin): post content on the Blog Editor field type" changed post content from `type: mlmarkdown` to `type: mlblogeditor` (+ htmlColumn, uploadFolder); no other post field is mlmarkdown. 47f3032 (16.2-02) extended the same test with bundle checks but did not touch the mlmarkdown assertion. The root package TestMain needs testcontainers, so the test never ran on the no-Docker 16.2 execution host (16.2-02-SUMMARY line 263/278/293). + implication: stale assertion contradicting an intended 16.2 change; same masking mechanism (Docker-only suite never executed before UAT) + +- timestamp: 2026-10-10T17:22:00+02:00 + checked: static diff (python over security_coverage_test.go phase09Routes and the conformCase literal keys) + found: inventory non-spa 56, cases 54; uncovered = ['GET /field-types', 'POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload']; not inventoried = [] + implication: exactly two cases are missing + +- timestamp: 2026-10-10T17:23:00+02:00 + checked: cabana run on this Docker host: `go test -json -count=1 -timeout 15m -skip '^TestPhase10OpenAPIConformance$' ./modules/cabana` + found: package PASS (77s), 291 top-level tests passed, no fails, no skips, no "no tests to run". TestFieldTypeSavePostgres PASS (real SELECT ... FOR UPDATE on ft_notes in its output), TestListRelationSearchPostgres PASS, TestMLOpenAPIConformance PASS. + implication: TestPhase10OpenAPIConformance is the only cabana failure in the postgres stage + +- timestamp: 2026-10-10T17:24:00+02:00 + checked: `go test -count=3 -timeout 10m -run '^TestPhase10OpenAPIConformance$' ./modules/cabana` + found: FAIL 3/3: run1 "POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload has no conformance case", run2 "GET /field-types has no conformance case", run3 upload again + implication: deterministic failure; which key is named depends on map iteration order + +- timestamp: 2026-10-10T17:25:00+02:00 + checked: 16.2-01-PLAN.md tasks 2/3 and 16.2-PATTERNS.md line 24/109 + found: both name admin_openapi.go + phase09Routes (+ check-admin-openapi.sh) as the places to register a new admin route; neither names openapi_conformance_test.go. Task automated verifies run only TestPhase09ContractInventory/TestPhase09PermissionMatrix (static), which pass. + implication: the route-registration checklist given to the executor was incomplete — the third location of the invariant was not in it + +- timestamp: 2026-10-10T17:26:00+02:00 + checked: compileFieldTypes (field_type.go ~160-190), conformPlugin, admin.json schemas + found: a registered type needs an element under the plugin's widget tag prefix and, for scripts/styles, pact.AdminAssets with the files in AdminFS (conformPlugin already has AdminFS with js/css assets). FieldTypesResponse decodes {"data":[...],"meta":{}} (SuccessMeta all-omitempty). Upload 201 is Envelope-cabana_FieldUploadResponse with multipart part "file". + implication: fix-direction constraints for the plan (see Resolution) + +## Resolution + +root_cause: "(1) Phase 16.2-01 added GET /field-types (717d46e) and POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload (8fa92db) to the admin route inventory (phase09Routes) and admin.json but added no conformCase to TestPhase10OpenAPIConformance (modules/cabana/openapi_conformance_test.go), whose exhaustiveness check (line 416-419) fails on whichever of the two it meets first; (2) sm-journal-plugin TestJournalSVGEmbed (plugin_test.go:122-123, in PG_JOURNAL_REQUIRE) still asserts `type: mlmarkdown` in models/post/fields.yaml after 16.2-03 (fbd1c85) changed post content to `type: mlblogeditor`; (3) contributing: both checks are static but live in testcontainers-only binaries (newConformEnv -> adminGorm runs before the inventory check; the Journal root TestMain starts Postgres), the 16.2 execution host had no Docker and the summaries filed every cabana/Journal-root failure as docker.sock noise, check-phase16.2.sh's go-stage FRAMEWORK_RUN omits TestPhase10OpenAPIConformance, and the 16.2-01 PLAN/PATTERNS route checklist omitted openapi_conformance_test.go." +fix: "" +verification: "" +files_changed: [] +suggested_fix_direction: | + - Add two conformCases. GET /field-types: status 200, ref "cabana.FieldTypesResponse", decode into[cabana.FieldTypesResponse](); register a plugin field type on the conform fixture (conformPlugin implements pact.HasFormFieldTypes; element under its widget tag prefix; a script already in conformFS) so the list is non-empty and the ?v= asset URL can be checked with assertPluginAsset. POST .../fields/{field}/upload: status 201, ref "cabana.Envelope-cabana_FieldUploadResponse", decode into[cabana.Envelope[cabana.FieldUploadResponse]](); a gadget fields.yaml field of that alias whose type has Upload, multipart part named "file", sent with requireAjax headers. Mind that a plugin-typed field on the gadget form also runs Decode/Load in the existing create/update/show cases — make them benign or give conformGadget a column for it. + - Fix TestJournalSVGEmbed to assert the intended `type: mlblogeditor` (optionally htmlColumn: content_html) for post content. + - Recurrence guard: split the static inventory check out of the Docker setup (e.g. move `env := newConformEnv(t)` after the coverage loop, or a package-level conformCaseKeys + a Docker-free TestPhase10ConformanceInventory) and add it to FRAMEWORK_RUN in check-phase16.2.sh (and later gates); add openapi_conformance_test.go to the "new admin route" checklist next to phase09Routes and admin_openapi.go.