docs(18.1-02): complete pgsub core plan

.planning/phases/18.1-payment-gateway-subscriptions/18.1-02-SUMMARY.md

@@ -0,0 +1,282 @@
+---
+phase: 18.1-payment-gateway-subscriptions
+plan: 02
+subsystem: payments
+tags: [subscriptions, pgsub, postgres, jsonb, gorm, festival, registry, testcontainers]
+
+# Dependency graph
+requires:
+  - phase: 18-payment-gateway
+    provides: the operator registry pattern (bootOperators over party.Activated), typed events with Tx, the plugin repo layout and harness, sm-paymentgateway-plugin (Requires)
+  - phase: 18.1-01
+    provides: nothing used directly (framework compiled through replace ../summercms.go; tag v0.1.26 held)
+provides:
+  - "new repository sm-paymentgatewaysubscriptions-plugin (module git.golem15.com/golem15/sm-paymentgatewaysubscriptions-plugin, package pgsub, plugin golem15.paymentgatewaysubscriptions), pushed to git.golem15.com at aad4647"
+  - "idempotent v2.4.0 baseline 202610150001_create_golem15_pgsub_baseline with JSONB JSON columns, and 202610150002_add_stripe_customer_id_to_users"
+  - "contracts.Gateway (incl. SyncPlanToGateway), Subscriber/SubscriberResolver, build-time Subscribers and Gateways registries with HasSubscribables/HasSubscriptionGateways"
+  - "classes.Service: Apply with the PHP event diff, ApplyDeleted, ApplyInvoice, ClaimEvent, EventProcessed and the D-13 service API; classes.UserSubscribable"
+  - "six typed events with the PHP golem15.pgsub.* names"
+affects: [18.1-03, 18.1-04, 18.1-06, sm-pgstripe-plugin, vue-paymentgateway-layer]
+
+# Actuals (#2632) - chars/4 over the realized diff (whole new repo, go.sum included)
+actuals:
+  tokens: 47958
+  tasks: 5
+  commits: 3
+plan_head_before: none (new repository, root commit 11a1730)
+plan_head_after: aad4647dcdf4c792f9fef9d50f34db76e4c18272
+commits: 3
+
+tech-stack:
+  added: []
+  patterns:
+    - "Gateway-neutral core: gateways map their payload to contracts.Update and call Service.Apply; the gateway never writes the database"
+    - "Build-time morph registry collected in Boot from party.Activated (HasSubscribables, HasSubscriptionGateways), duplicates fail boot naming both plugins"
+    - "Claim-first processed-event marker (INSERT ... ON CONFLICT (event_id) DO NOTHING) inside the applying transaction"
+    - "lagoon.Jsonable over JSONB columns (no adapter needed)"
+
+key-files:
+  created:
+    - ../sm-paymentgatewaysubscriptions-plugin/plugin.go
+    - ../sm-paymentgatewaysubscriptions-plugin/boot.go
+    - ../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go
+    - ../sm-paymentgatewaysubscriptions-plugin/contracts/subscriber.go
+    - ../sm-paymentgatewaysubscriptions-plugin/contracts/registry.go
+    - ../sm-paymentgatewaysubscriptions-plugin/models/subscription.go
+    - ../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go
+    - ../sm-paymentgatewaysubscriptions-plugin/models/processed_webhook_event.go
+    - ../sm-paymentgatewaysubscriptions-plugin/models/feature_list.go
+    - ../sm-paymentgatewaysubscriptions-plugin/models/registry.go
+    - ../sm-paymentgatewaysubscriptions-plugin/events/events.go
+    - ../sm-paymentgatewaysubscriptions-plugin/classes/service.go
+    - ../sm-paymentgatewaysubscriptions-plugin/classes/apply.go
+    - ../sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go
+    - ../sm-paymentgatewaysubscriptions-plugin/updates/registry.go
+    - ../sm-paymentgatewaysubscriptions-plugin/updates/202610150001_create_golem15_pgsub_baseline.go
+    - ../sm-paymentgatewaysubscriptions-plugin/updates/202610150002_add_stripe_customer_id_to_users.go
+    - ../sm-paymentgatewaysubscriptions-plugin/config/config.yaml
+    - ../sm-paymentgatewaysubscriptions-plugin/lang/en/lang.yaml
+    - ../sm-paymentgatewaysubscriptions-plugin/lang/pl/lang.yaml
+    - ../sm-paymentgatewaysubscriptions-plugin/README.md
+    - ../sm-paymentgatewaysubscriptions-plugin/testmain_test.go
+    - ../sm-paymentgatewaysubscriptions-plugin/harness_test.go
+    - ../sm-paymentgatewaysubscriptions-plugin/fixture_plugin_test.go
+    - ../sm-paymentgatewaysubscriptions-plugin/tracer_test.go
+    - ../sm-paymentgatewaysubscriptions-plugin/core_smoke_test.go
+  modified: []
+
+key-decisions:
+  - "Baseline JSON columns (stripe_prices, currency_prices, features) are JSONB (human reply baseline-jsonb); lagoon.Jsonable scans and writes JSONB unchanged"
+  - "Gateways never write the database; the service stores the customer id in its own short transaction and saves gateway-mutated rows"
+  - "The service writes cancel_at_period_end locally after a successful cancel at period end, without events"
+  - "Plan lookup by price id is exact (stripe_price_id, then stripe_prices values), never LIKE"
+  - "Plan rules use the lagoon model rule set (size:3 as between:3,3, code max:50); no BeforeValidate fallback (cabana CRUD maps a BeforeValidate error to a capability error, not a 422)"
+  - "Repository created with tea and main pushed at aad4647 (human reply create-and-push)"
+
+patterns-established:
+  - "pgsub fixture host: golem15.user + golem15.paymentgateway + pgsub + acme.members (user subscriber, recording fake stripe gateway, event recorder published on the app)"
+
+requirements-completed: [SC-1, SC-2, SC-3, D-01, D-02, D-03, D-04, D-05, D-06, D-08, D-09, D-13, D-16, D-22]
+
+coverage:
+  - id: D1
+    description: "Idempotent v2.4.0 baseline with JSONB columns, the stripe_status CHECK, the RESTRICT FK, no dormant tables; users.stripe_customer_id only when missing"
+    requirement: SC-1
+    verification:
+      - kind: integration
+        ref: "core_smoke_test.go#TestSubscriptionsCoreSmoke/baseline_idempotent, users_column_preexisting; tracer_test.go#TestPlanJSONBColumns"
+        status: pass
+    human_judgment: false
+  - id: D2
+    description: "Build-time subscriber and gateway registries; duplicate alias fails boot; unknown alias fails loudly; built-in user subscriber checkout stores the customer id"
+    requirement: SC-1
+    verification:
+      - kind: integration
+        ref: "tracer_test.go#TestSubscriptionsCheckout; core_smoke_test.go#TestSubscriptionsCoreSmoke/duplicate_alias, unknown_alias"
+        status: pass
+    human_judgment: false
+  - id: D3
+    description: "Gateway-neutral Apply with the PHP event diff, ApplyDeleted, ApplyInvoice and the processed-event claim"
+    requirement: SC-2
+    verification:
+      - kind: integration
+        ref: "tracer_test.go#TestSubscriptionsTracer; core_smoke_test.go#TestSubscriptionsCoreSmoke/event_diff, invoices, claim"
+        status: pass
+    human_judgment: false
+  - id: D4
+    description: "Service API: plans, latest/active, access and grace, features, checkout, portal, cancel, sync subscription, sync plan; grace_period_days config"
+    requirement: SC-3
+    verification:
+      - kind: integration
+        ref: "core_smoke_test.go#TestSubscriptionsCoreSmoke/grace, sync_plan_free, sync_plan_paid, cancel_at_period_end, service_api, plan_rules"
+        status: pass
+    human_judgment: false
+
+duration: 45min
+completed: 2026-10-14
+status: complete
+---
+
+# Phase 18.1 Plan 02: pgsub Core Summary
+
+**New repository `sm-paymentgatewaysubscriptions-plugin` with the gateway-neutral subscriptions core. It holds an idempotent JSONB v2.4.0 baseline, build-time subscriber and gateway registries, the built-in `user` subscriber and `classes.Service`. The service covers Apply with the PHP event diff, invoices, the claim-first event marker and the D-13 service API. Postgres tests prove all of it, and main is pushed to git.golem15.com at `aad4647`.**
+
+## Performance
+
+- **Duration:** about 45 min, including the wait for the Task 5 reply
+- **Started:** 2026-10-14T14:41:36Z
+- **Completed:** 2026-10-14
+- **Tasks:** 5 (2 checkpoints, 3 implementation)
+- **Files:** 30 in the new repository
+
+## Repository
+
+- Path: `../sm-paymentgatewaysubscriptions-plugin`, module `git.golem15.com/golem15/sm-paymentgatewaysubscriptions-plugin`, branch `main`.
+- go.mod requires `summercms v0.1.25` with `replace ../summercms.go` (v0.1.26 is held) and `sm-paymentgateway-plugin v0.0.0` with `replace ../sm-paymentgateway-plugin`. go.work also replaces `sm-user-plugin` with `../sm-summercmsio-app/plugins/golem15/user`. No new third-party module was added.
+- Remote: `git@git.golem15.com:golem15/sm-paymentgatewaysubscriptions-plugin.git`, created with `tea repos create --owner golem15 --name sm-paymentgatewaysubscriptions-plugin`. `main` was pushed with `-u`.
+- **Pushed hash:** `aad4647dcdf4c792f9fef9d50f34db76e4c18272`; `rev-parse main` equals `rev-parse origin/main`. No tags were pushed, and the framework repo was not pushed.
+
+## Task Commits
+
+| Task | Name | Commit | Repo |
+|------|------|--------|------|
+| 1 | Baseline confirmation (checkpoint:decision) | none (no files) | n/a |
+| 2 | Tracer: plugin, baseline, Apply, activated event, access | `11a1730` | sm-paymentgatewaysubscriptions-plugin |
+| 3 | Registries, Gateway contract, built-in user subscriber, checkout | `5388ffb` | sm-paymentgatewaysubscriptions-plugin |
+| 4 | Complete core: checks, event diff, invoices, claim, service API, config, copy, smoke | `aad4647` | sm-paymentgatewaysubscriptions-plugin |
+| 5 | Create the remote and push (checkpoint:decision) | pushed `aad4647` | remote |
+
+## Task 1: baseline sheet and reply
+
+**Reply (verbatim):** "baseline-jsonb"
+
+**Migration `202610150001_create_golem15_pgsub_baseline`.** Every statement uses `IF NOT EXISTS`. The rollback drops the three tables IF EXISTS in dependency order.
+
+- **`golem15_pgsub_subscription_plans`:**
+  - Columns: id SERIAL PK, code VARCHAR(50) NOT NULL, name VARCHAR(255) NOT NULL, description TEXT NULL, stripe_price_id VARCHAR(255) NULL, stripe_product_id VARCHAR(255) NULL, **stripe_prices JSONB NULL**, price_cents INTEGER NOT NULL DEFAULT 0, **currency_prices JSONB NULL**, currency VARCHAR(3) NOT NULL DEFAULT 'EUR', billing_interval VARCHAR(20) NOT NULL DEFAULT 'month', **features JSONB NULL**, max_quotes_per_month INTEGER NULL, lead_fee_required BOOLEAN NOT NULL DEFAULT FALSE, is_active BOOLEAN NOT NULL DEFAULT TRUE, sort_order INTEGER NOT NULL DEFAULT 0, created_at and updated_at TIMESTAMPTZ NULL.
+  - Indexes: `golem15_pgsub_subscription_plans_code_unique`, `_code_index`, `_is_active_index`, `_sort_order_index`.
+- **`golem15_pgsub_subscriptions`:**
+  - Columns: id SERIAL PK, subscriber_type VARCHAR(255) NULL, subscriber_id INTEGER NULL, gateway VARCHAR(20) NOT NULL DEFAULT 'stripe', subscription_plan_id INTEGER NOT NULL, stripe_subscription_id VARCHAR(255) NULL, stripe_customer_id VARCHAR(255) NULL, stripe_status VARCHAR(255) NOT NULL DEFAULT 'incomplete', current_period_start, current_period_end, canceled_at, ended_at and trial_ends_at TIMESTAMPTZ NULL, cancel_at_period_end BOOLEAN NOT NULL DEFAULT FALSE, created_at and updated_at TIMESTAMPTZ NULL.
+  - CHECK `golem15_pgsub_subscriptions_stripe_status_check` over the eight PHP statuses.
+  - FK `golem15_pgsub_subscriptions_subscription_plan_id_foreign` ON DELETE RESTRICT.
+  - Indexes: `golem15_pgsub_subscriptions_stripe_subscription_id_unique`, `_subscription_plan_id_index`, `_stripe_status_index`, `_current_period_end_index`, `pgsub_subscriber_index`, `pgsub_gateway_index`.
+- **`golem15_pgsub_processed_webhook_events`:**
+  - Columns: id BIGSERIAL PK, event_id VARCHAR(255) NOT NULL, event_type VARCHAR(255) NULL, processed_at, created_at and updated_at TIMESTAMPTZ NULL.
+  - Indexes: `golem15_pgsub_processed_webhook_events_event_id_unique`, `_event_id_index`.
+
+**Migration `202610150002_add_stripe_customer_id_to_users`:**
+- `ALTER TABLE users ADD COLUMN IF NOT EXISTS stripe_customer_id VARCHAR(255) NULL`
+- `CREATE INDEX IF NOT EXISTS users_stripe_customer_id_index`
+- The rollback is a no-op: the shared column may predate the plugin.
+
+**Deliberately absent:** the two dormant usage and featured-placement tables (D-16), the starter/professional/premium seed, the professionals-table ALTER and the professional_id to polymorphic conversion.
+
+**The one difference between the options:** the type of stripe_prices, currency_prices and features: TEXT under baseline-research, JSONB under baseline-jsonb (chosen).
+
+## Task 5: remote and push
+
+**Reply (verbatim):** "create-and-push"
+
+Before acting I checked the state: `main` at `aad4647`, clean tree, no remote, and `git ls-remote` answered "Cannot find repository". tea then created `golem15/sm-paymentgatewaysubscriptions-plugin` with admin permission. `origin` was added and `git push -u origin main` created the branch. **Pushed hash `aad4647dcdf4c792f9fef9d50f34db76e4c18272`**, and main equals origin/main.
+
+## Accomplishments
+
+- **Plugin.** `golem15.paymentgatewaysubscriptions` (package `pgsub`) requires `golem15.paymentgateway` and `golem15.user`. It embeds config and lang and runs two boot steps: the registries, then the service.
+- **Models.** `Subscription` and `SubscriptionPlan` carry the PHP fillables, rules, scopes and checks. `ProcessedWebhookEvent` is the marker model. `FeatureList` stores a JSON list and also decodes the legacy truthy map. The eight statuses and two gateways use the verbatim PHP strings.
+- **Contracts.** `Gateway`, `Subscriber` and `SubscriberResolver`, `CheckoutOptions` and `CheckoutSession`, and `Update`, `InvoiceUpdate` and `Invoice`. The `Subscribers` and `Gateways` registries come with their sentinel errors.
+- **Events.** Six typed events carry the literal `golem15.pgsub.*` names. They fire inside the transaction with `Tx`.
+- **Apply.** The PHP event diff runs in order (reactivated, then the status-unchanged stop, then activated, past_due, canceled). `ApplyDeleted` always fires canceled. `ApplyInvoice` fires with a PII-free invoice. `ClaimEvent` uses ON CONFLICT DO NOTHING, and `EventProcessed` is the cheap check.
+- **Service API.** `ActivePlans`, `ActivePlanByCode`, `Latest`, `Active`, `HasActiveSubscription`, `OnPlan`, `CurrentPlanCode`, `Subscriber`, `Checkout`, `Portal`, `Cancel`, `HasAccess`, `InGracePeriod`, `HasFeature`, `SyncSubscription` and `SyncPlan`.
+- **Config.** `grace_period_days` defaults to 7. A negative or non-numeric value fails boot.
+- **Copy.** EN and PL lang files: plugin name and description, `statuses.*`, `gateways.*`, `errors.*`.
+- **README.** Covers the API reference, configuration and the differences from PHP. It names no consuming application.
+
+## Exported identifiers
+
+- **pgsub:**
+  - `PluginID`, `ConfigGracePeriodDays`
+  - `Plugin`, with accessors `Service`, `Subscribers`, `Gateways`
+- **contracts:**
+  - `Gateway` (Identifier, GetOrCreateCustomer, CreateCheckoutSession, CreateManagementSession, CancelSubscription, SyncSubscription, SyncPlanToGateway, SupportsCurrency); `CheckoutOptions{Gateway, Currency, Metadata}`; `CheckoutSession{URL, SessionID}`
+  - `Update`, `InvoiceUpdate`, `Invoice`
+  - `Subscriber`, `SubscriberResolver`
+  - `Subscribers`, `Gateways`, `NewSubscribers`, `NewGateways`; `HasSubscribables`, `HasSubscriptionGateways`
+  - errors `ErrUnknownSubscriberType`, `ErrDuplicateAlias`, `ErrDuplicateGateway`, `ErrGatewayNotInstalled`, `ErrSubscriberNotFound`
+- **models:**
+  - `Subscription`, `SubscriptionPlan`, `ProcessedWebhookEvent`, `FeatureList`
+  - `StatusIncomplete` … `StatusPaused` and `Statuses`; `GatewayStripe`, `GatewayGooglePlay` and `Gateways`; `BillingIntervalMonth`, `BillingIntervalYear`
+  - `SubscriptionModelName`, `SubscriptionPlanModelName`, `ProcessedWebhookEventModelName`
+  - scopes `ForSubscriber`, `ScopeActive`, `ScopePastDue`, `ScopeCanceled`, `ScopeFromGateway`, `ScopeActivePlans`, `ScopeOrdered`
+  - model methods `IsActive`, `InGracePeriod`, `HasAccess`, `HasFeature`, `WillCancelAtPeriodEnd`, `DaysUntilRenewal`, `OnTrial`, `IsFromGateway`, `IsStripeSubscription`, `IsGooglePlaySubscription`, `PriceCentsFor`, `StripePriceIDFor`, `TierLevel`, `Rules`, `Fillable`, `MorphName`
+- **events:** `SubscriptionActivated`, `SubscriptionCanceled`, `SubscriptionPastDue`, `SubscriptionReactivated` (embedding `SubscriptionEvent`); `InvoicePaid`, `InvoicePaymentFailed` (embedding `InvoiceEvent`); the six `...EventName` constants.
+- **classes:**
+  - `Service` with the methods above, plus `Apply`, `ApplyDeleted`, `ApplyInvoice`, `ClaimEvent`, `EventProcessed`, `Gateway`, `Gateways`, `GracePeriodDays` and the `Now` field
+  - `New`, `Options`, `DefaultGracePeriodDays`, `ErrNoCustomer`, `UserSubscribable`, `UserAlias`
+- **Signatures plan 03 relies on:**
+  - `ApplyDeleted(ctx, tx, gateway, subscriptionID) (*models.Subscription, error)`
+  - `ApplyInvoice(ctx, tx, contracts.InvoiceUpdate) (*models.Subscription, error)`
+  - `ClaimEvent(ctx, tx, eventID, eventType) (bool, error)`
+  - `EventProcessed(ctx, db, eventID) (bool, error)`
+  - `SyncPlan(ctx, tx, plan) (map[string]string, error)`
+
+## Tests
+
+All Docker tests ran on testcontainers Postgres 16.
+- `go vet ./...` and `go test -short ./...` pass. `go list -deps ./...` contains no stripe-go.
+- Task 2 verify passed (`TestSubscriptionsTracer`). The tracer gate re-ran the verify end to end before expanding.
+- Task 3 verify: 2 of 2 pass (`TestSubscriptionsTracer`, `TestSubscriptionsCheckout`).
+- Task 4 verify: 3 of 3 pass (`TestSubscriptionsCoreSmoke` with 13 subtests, the tracer and the checkout test). The full `go test ./...` passes.
+- **JSONB:** `TestPlanJSONBColumns` checks:
+  - the round trip, with an unset value read back as SQL NULL;
+  - `jsonb_typeof` answers `object`;
+  - malformed JSON is rejected;
+  - the exact stripe_prices lookup (price_pln_1 does not match price_pln_10).
+- **Acceptance greps:**
+  - 16 `IF NOT EXISTS` in the baseline;
+  - 0 dormant table names;
+  - 6 event names;
+  - the Requires line;
+  - no application names;
+  - `ON CONFLICT (event_id) DO NOTHING` present, 0 `LIKE`;
+  - 18 of 18 Service methods;
+  - no `FEATURE_` or `UnlimitedQuotes`;
+  - no `.Save(` in the user subscriber;
+  - `## API reference` present.
+
+## Deviations from Plan
+
+### Decisions and auto-fixed issues
+
+1. **[Task 1 decision] JSONB columns.** This departs from the RESEARCH TEXT DDL and the lagoon.Jsonable TEXT convention. No adaptation was needed: `lagoon.Jsonable[map[string]string]` and `[map[string]int64]` scan and write JSONB through pgx/GORM as they are, and `FeatureList` accepts `[]byte` and `string`. Proven on Postgres by TestPlanJSONBColumns and the `baseline_idempotent` data-type check.
+2. **[Rule 2] Apply input checks.** Apply refuses a status or gateway outside the PHP value lists. It also checks the subscriber/plan pair on every save, not only on new rows, which matches PHP's model validation.
+3. **Plan rules have no BeforeValidate fallback.** The lagoon model rule set covers every PHP rule: size:3 becomes `between:3,3`, `string` is implicit, and `exists` is left to the FK. Cabana CRUD maps a BeforeValidate error to a CapabilityError, not a field 422. I added `max:50` on code for the column width.
+4. **Lang keys follow the plan.** They use `statuses.*` and `gateways.*`, as plans 04 and 05 expect. PHP used `status.*`.
+5. **NULL ordering.** Latest and Active order by `created_at DESC NULLS LAST, id DESC`, matching MySQL's NULL ordering for PHP `latest()`.
+6. **Additive identifiers.** `ErrSubscriberNotFound`, `ErrNoCustomer` (in classes), `GracePeriodDays()`, the plugin's registry accessors, the BillingInterval constants and the Statuses/Gateways slices.
+7. **Which gateway a call uses.** Portal and SyncPlan use the `stripe` gateway (no gateway parameter). Cancel and SyncSubscription use `sub.Gateway`, falling back to stripe when empty.
+8. **Test additions.**
+   - `TestPlanJSONBColumns`;
+   - the `acme.dupusers` duplicate-alias fixture;
+   - extra smoke subtests: `sync_plan_paid` (product id kept when a price step fails), `service_api` (including the legacy feature-map decode), `plan_rules`, and the grace env override (0 days, -1 fails boot).
+9. **User subscriber name fallback.** The built-in `user` subscriber falls back to the e-mail, because the users table has no username column (RESEARCH Pattern 7).
+
+**Total deviations:** one decided change (JSONB), one Rule 2 addition and several additive identifiers. None changes scope.
+
+## Issues Encountered
+
+- `run()` first returned `gormigrate.MigrateFunc`, which a Rollback field cannot take. It now returns an unnamed `func(*gorm.DB) error`.
+- A smoke subtest reused a loaded model in `First`, which added a second id condition. I fixed the test.
+
+## Known Stubs
+
+None.
+
+## Next Phase Readiness
+
+- Plan 03 can point `sm-pgstripe-plugin/subscriptions` at the committed local repository (replace) or at the pushed remote. The contracts it consumes are listed above.
+- Plan 04 builds the admin screens and the customer API on `classes.Service`, the models' rules and the `statuses.*` and `gateways.*` copy.
+
+## Self-Check: PASSED
+
+- All 30 files exist in `../sm-paymentgatewaysubscriptions-plugin`, and the tree is clean.
+- Commits `11a1730`, `5388ffb` and `aad4647` exist on `main`. `origin/main` = `aad4647dcdf4c792f9fef9d50f34db76e4c18272`.