docs(05-03): complete secrets-slice encrypted credentials plan

- AES-256-GCM Encrypted, key:generate, Laravel helper, organisations, OAuth tables
2026-09-18 19:43:02 +02:00
parent 8d9109a771
commit 76852e2d55
1 changed files with 214 additions and 0 deletions
--- a/.planning/phases/05-data-layer-full-fidelity/05-03-SUMMARY.md
+++ b/.planning/phases/05-data-layer-full-fidelity/05-03-SUMMARY.md
@@ -0,0 +1,214 @@
+---
+phase: 05-data-layer-full-fidelity
+plan: 03
+subsystem: database
+tags: [aes-gcm, hkdf, encrypted, gormigrate, oauth, credentials, organisations]
+
+requires:
+  - phase: 05-data-layer-full-fidelity
+    provides: Winter-directory registries, lagoon.Fill, lagoon.Jsonable, HasHidden
+provides:
+  - lagoon.Encrypted AES-256-GCM Scanner/Valuer with stdlib HKDF and previous_keys fallback
+  - summer key:generate and standalone DecryptLaravelPayload (unwired from Scan/Value)
+  - golem15_user_organisations structure-only table in the user plugin
+  - api_tokens, 3 OAuth tables, 4 credential tables and models with Hidden/json:"-"/Encrypted
+affects: [05-04-attachments, 05-05-remaining-models, 05-06-tests, 07-user-plugin, 08-oauth]
+
+tech-stack:
+  added: []
+  patterns:
+    - Encrypted plaintext is unexported; Reveal() is the only greppable accessor
+    - OpenFromApp loads app.key once and PublishEncryptionKeys so rows do not re-read config
+    - Organisations owned by golem15.user even though only Fonoteka credentials FK this phase
+
+key-files:
+  created:
+    - lagoon/encrypted.go
+    - lagoon/encrypted_test.go
+    - lagoon/keygen.go
+    - lagoon/keygen_test.go
+    - lagoon/laravel_decrypt.go
+    - lagoon/laravel_decrypt_test.go
+    - plugins/golem15/user/updates/10_organisations.go
+    - plugins/golem15/user/models/organisation.go
+    - plugins/golem15/fonoteka/updates/11_secrets_slice.go
+    - plugins/golem15/fonoteka/models/api_token.go
+    - plugins/golem15/fonoteka/models/oauth_client.go
+    - plugins/golem15/fonoteka/models/oauth_auth_code.go
+    - plugins/golem15/fonoteka/models/oauth_refresh_token.go
+    - plugins/golem15/fonoteka/models/user_ai_credential.go
+    - plugins/golem15/fonoteka/models/org_ai_credential.go
+    - plugins/golem15/fonoteka/models/user_discogs_credential.go
+    - plugins/golem15/fonoteka/models/org_discogs_credential.go
+  modified:
+    - lagoon/commands.go
+    - lagoon/connection.go
+    - fonoteka.go/config/app.yaml
+    - fonoteka.go/parity/migrate_test.go
+    - fonoteka.go/plugins/golem15/user/go.mod
+
+key-decisions:
+  - "Column keys are HKDF-SHA256 derived via Go 1.27 crypto/hkdf with info summercms.lagoon.encrypted.v1; no golang.org/x/crypto"
+  - "Ciphertext is 1-byte format/key-id | 12-byte nonce | GCM seal, stored base64 in text columns"
+  - "OpenFromApp calls LoadAppKey+PublishEncryptionKeys once per boot; empty/short/undecodable app.key fails with SUMMER_APP__KEY"
+  - "golem15_user_organisations is owned by the user plugin; Phase 5 ships no users-table ALTER (D-03 deviation, Phase 7 AUTH-02)"
+  - "DecryptLaravelPayload is cutover-import-only and is never called from Encrypted Scan/Value"
+
+patterns-established:
+  - "NewEncrypted(plaintext) for writes; Reveal() for reads; MarshalJSON/String/GoString always [redacted]"
+  - "One Register call per model/migration file init(); plugin.go is not edited"
+  - "Secret columns are json:\"-\" plus Hidden(); credential secrets are additionally lagoon.Encrypted"
+
+requirements-completed: [DATA-07, DATA-09]
+
+duration: 15min
+completed: 2026-09-18
+---
+
+# Phase 5 Plan 03: Secrets slice Summary
+
+**AES-256-GCM `lagoon.Encrypted` with stdlib HKDF and `app.previous_keys` rotation, `summer key:generate`, a standalone Laravel CBC decrypt helper, user-plugin `golem15_user_organisations`, and Fonoteka api_tokens/OAuth/credential tables**
+
+## Performance
+
+- **Duration:** 15 min
+- **Started:** 2026-09-18T17:26:51Z
+- **Completed:** 2026-09-18T17:41:45Z
+- **Tasks:** 3
+- **Files modified:** 28
+
+## Accomplishments
+
+- `lagoon.Encrypted` round-trips AES-256-GCM ciphertext keyed by HKDF from `app.key`, redacts on JSON/`String`/`%#v`, and decrypts rows after primary rotation via `app.previous_keys`
+- Missing, short, or undecodable `app.key` fails loudly mentioning `SUMMER_APP__KEY`; `key:generate` prints a fresh 32-byte base64 key using `crypto/rand`
+- `DecryptLaravelPayload` decrypts a real Laravel AES-256-CBC+HMAC fixture and is not on the live Scan/Value path
+- `golem15_user_organisations` (slug/name/description, structure only) is created by the user plugin; no users-table ALTER ships this phase
+- Three Fonoteka migrations create api_tokens, 3 OAuth tables, and 4 credential tables; hash/secret columns are `json:"-"`/`Hidden()`, and `api_key`/`token` are `lagoon.Encrypted`
+
+## Task Commits
+
+Each task was committed atomically:
+
+1. **Task 1 RED: Encrypted tests** - `06bad37` (test) in summercms.go
+2. **Task 1 GREEN: Encrypted implementation** - `8d9109a` (feat) in summercms.go
+3. **Task 2: create_organisations in user plugin** - `c291280` (feat) in fonoteka.go
+4. **Task 3: secrets-slice migrations and models** - `f3bf0dd` (feat) in fonoteka.go
+
+**Plan metadata:** pending (this file)
+
+## Files Created/Modified
+
+- `lagoon/encrypted.go`, `lagoon/encrypted_test.go` — AES-256-GCM cast, HKDF, previous_keys, redaction
+- `lagoon/keygen.go`, `lagoon/keygen_test.go` — `key:generate`
+- `lagoon/laravel_decrypt.go`, `lagoon/laravel_decrypt_test.go` — cutover-only Laravel payload helper
+- `lagoon/commands.go` — RuntimeCommands appends KeyGenerateCommand
+- `lagoon/connection.go` — OpenFromApp publishes encryption keys from `app.key`
+- `fonoteka.go/config/app.yaml` — documented empty `app.key` placeholder
+- `fonoteka.go/plugins/golem15/user/updates/10_organisations.go`, `models/organisation.go` — FK target table
+- `fonoteka.go/plugins/golem15/fonoteka/updates/11_secrets_slice.go` — api_tokens / OAuth / credentials
+- `fonoteka.go/plugins/golem15/fonoteka/models/{api_token,oauth_*,*_credential}.go` — 8 models
+- `fonoteka.go/parity/migrate_test.go` — 11 Fonoteka IDs, 2 user IDs, secrets rollback coverage
+
+## Decisions Made
+
+- Column keys use Go 1.27 `crypto/hkdf` (info `summercms.lagoon.encrypted.v1`); no new dependency
+- Ciphertext format is version nibble `0x1` plus key-id, 12-byte nonce, GCM ciphertext+tag, base64 for `text` columns
+- `OpenFromApp` is the boot wiring site for `LoadAppKey` + `PublishEncryptionKeys`; Scan/Value use the package-level key set
+- Organisations belong to `golem15.user`; Phase 5 does not ALTER `users` (confirmed D-03 deviation, Phase 7 AUTH-02)
+- `DecryptLaravelPayload` stays unwired from the live cast (D-10)
+
+## Deviations from Plan
+
+### Auto-fixed Issues
+
+**1. [Rule 2 - Missing Critical] Write-path constructor for Encrypted**
+- **Found during:** Task 1 (GREEN)
+- **Issue:** Scan/Value/Reveal cannot set plaintext from another package
+- **Fix:** Exported `NewEncrypted(plaintext string)`
+- **Files modified:** `lagoon/encrypted.go`
+- **Verification:** `TestEncryptedRoundTrip`
+- **Committed in:** `8d9109a`
+
+**2. [Rule 3 - Blocking] OpenFromApp now requires a valid app.key**
+- **Found during:** Task 1 (wiring PublishEncryptionKeys into OpenFromApp)
+- **Issue:** `TestOpenFromAppReadsDSN` had no `SUMMER_APP__KEY`
+- **Fix:** Test supplies a 32-byte base64 key via environ
+- **Files modified:** `lagoon/connection_test.go`
+- **Verification:** `go test ./lagoon/...`
+- **Committed in:** `8d9109a`
+
+**3. [Rule 3 - Blocking] RuntimeCommands name list omitted key:generate**
+- **Found during:** Task 1
+- **Issue:** `TestRuntimeCommandsRegisterBareAndColonNames` expected the old trio
+- **Fix:** Assert `key:generate` is registered
+- **Files modified:** `lagoon/migrations_test.go`
+- **Verification:** that test
+- **Committed in:** `8d9109a`
+
+**4. [Rule 2 - Missing Critical] Organisations columns from PHP**
+- **Found during:** Task 2 (reading `create_organisations_table.php`)
+- **Issue:** Plan listed id/name/timestamps as minimum; PHP also has unique `slug` and nullable `description`
+- **Fix:** Table and model include slug and description; no membership/role tables
+- **Files modified:** `plugins/golem15/user/updates/10_organisations.go`, `models/organisation.go`
+- **Verification:** `TestCreateOrganisationsMigration`
+- **Committed in:** `c291280`
+
+**5. [Rule 3 - Blocking] Existing migrate tests hardcoded ID counts**
+- **Found during:** Task 3
+- **Issue:** Parity tests expected 1 user ID and 8 Fonoteka IDs; last rollback targeted album_reservations
+- **Fix:** Expect 2 user IDs and 11 Fonoteka IDs; RollbackLast now drops credential tables first
+- **Files modified:** `fonoteka.go/parity/migrate_test.go`
+- **Verification:** `TestMigrateSeedsCanonicalGenres`, `TestSecretsSliceMigrationsUpDown`, `TestRollbackLastIsolatesFonoteka`
+- **Committed in:** `f3bf0dd`
+
+**6. [Rule 3 - Blocking] Acceptance grep for widen_users vs required doc comment**
+- **Found during:** Task 2
+- **Issue:** Plan required a comment naming the omitted users ALTER, which would fail `grep -rln widen_users`
+- **Fix:** Comment states the decision without the `widen_users` token
+- **Files modified:** `plugins/golem15/user/updates/10_organisations.go`
+- **Verification:** grep returns no matches
+- **Committed in:** `c291280`
+
+---
+
+**Total deviations:** 6 auto-fixed (2 missing critical, 4 blocking)
+**Impact on plan:** Required for a compiling write path, boot-time key loading, PHP column fidelity, and green existing migrate tests. No scope creep. `LoadAppKey` returns `[][]byte` previous keys rather than a concatenated blob so Scan can try each key.
+
+## Issues Encountered
+
+None beyond the auto-fixes above.
+
+## Authentication Gates
+
+None.
+
+## TDD Gate Compliance
+
+- RED: `06bad37` `test(05-03): add failing tests for Encrypted, key:generate, Laravel decrypt`
+- GREEN: `8d9109a` `feat(05-03): implement lagoon.Encrypted, key:generate, Laravel decrypt helper`
+- REFACTOR: not needed
+
+## Known Stubs
+
+- `fonoteka.go/config/app.yaml` `app.key: ""` is an intentional documented placeholder; empty fails boot once an encrypted-cast model runs / `OpenFromApp` loads keys. Operators set `SUMMER_APP__KEY` from `summer key:generate`.
+
+## User Setup Required
+
+None - no external service configuration required. Operators must set `SUMMER_APP__KEY` before serving or migrating through `OpenFromApp`.
+
+## Next Phase Readiness
+
+Ready for 05-04 (attachments). Encrypted credentials, OAuth tables, and the organisations FK target exist. zitadel/oidc storage remains Phase 8; organisation membership/roles remain Phase 7.
+
+## Self-Check: PASSED
+
+- Key files exist on disk (`lagoon/encrypted.go`, `keygen.go`, `laravel_decrypt.go`, user `10_organisations.go`, fonoteka `11_secrets_slice.go`, 8 models)
+- Commits `06bad37`, `8d9109a` (summercms.go) and `c291280`, `f3bf0dd` (fonoteka.go) exist
+- `go vet ./lagoon/...` and `go test ./lagoon/...` green in summercms.go
+- `go vet ./...` and `go test ./...` green in fonoteka.go
+- `DecryptLaravelPayload(` is called only from its file and test
+- `lagoon/encrypted.go` does not import `golang.org/x/crypto`; `keygen.go` does not import `math/rand`
+
+---
+*Phase: 05-data-layer-full-fidelity*
+*Completed: 2026-09-18*