diff --git a/.planning/phases/05-data-layer-full-fidelity/05-03-SUMMARY.md b/.planning/phases/05-data-layer-full-fidelity/05-03-SUMMARY.md new file mode 100644 index 0000000..de2c8a0 --- /dev/null +++ b/.planning/phases/05-data-layer-full-fidelity/05-03-SUMMARY.md @@ -0,0 +1,214 @@ +--- +phase: 05-data-layer-full-fidelity +plan: 03 +subsystem: database +tags: [aes-gcm, hkdf, encrypted, gormigrate, oauth, credentials, organisations] + +requires: + - phase: 05-data-layer-full-fidelity + provides: Winter-directory registries, lagoon.Fill, lagoon.Jsonable, HasHidden +provides: + - lagoon.Encrypted AES-256-GCM Scanner/Valuer with stdlib HKDF and previous_keys fallback + - summer key:generate and standalone DecryptLaravelPayload (unwired from Scan/Value) + - golem15_user_organisations structure-only table in the user plugin + - api_tokens, 3 OAuth tables, 4 credential tables and models with Hidden/json:"-"/Encrypted +affects: [05-04-attachments, 05-05-remaining-models, 05-06-tests, 07-user-plugin, 08-oauth] + +tech-stack: + added: [] + patterns: + - Encrypted plaintext is unexported; Reveal() is the only greppable accessor + - OpenFromApp loads app.key once and PublishEncryptionKeys so rows do not re-read config + - Organisations owned by golem15.user even though only Fonoteka credentials FK this phase + +key-files: + created: + - lagoon/encrypted.go + - lagoon/encrypted_test.go + - lagoon/keygen.go + - lagoon/keygen_test.go + - lagoon/laravel_decrypt.go + - lagoon/laravel_decrypt_test.go + - plugins/golem15/user/updates/10_organisations.go + - plugins/golem15/user/models/organisation.go + - plugins/golem15/fonoteka/updates/11_secrets_slice.go + - plugins/golem15/fonoteka/models/api_token.go + - plugins/golem15/fonoteka/models/oauth_client.go + - plugins/golem15/fonoteka/models/oauth_auth_code.go + - plugins/golem15/fonoteka/models/oauth_refresh_token.go + - plugins/golem15/fonoteka/models/user_ai_credential.go + - plugins/golem15/fonoteka/models/org_ai_credential.go + - plugins/golem15/fonoteka/models/user_discogs_credential.go + - plugins/golem15/fonoteka/models/org_discogs_credential.go + modified: + - lagoon/commands.go + - lagoon/connection.go + - fonoteka.go/config/app.yaml + - fonoteka.go/parity/migrate_test.go + - fonoteka.go/plugins/golem15/user/go.mod + +key-decisions: + - "Column keys are HKDF-SHA256 derived via Go 1.27 crypto/hkdf with info summercms.lagoon.encrypted.v1; no golang.org/x/crypto" + - "Ciphertext is 1-byte format/key-id | 12-byte nonce | GCM seal, stored base64 in text columns" + - "OpenFromApp calls LoadAppKey+PublishEncryptionKeys once per boot; empty/short/undecodable app.key fails with SUMMER_APP__KEY" + - "golem15_user_organisations is owned by the user plugin; Phase 5 ships no users-table ALTER (D-03 deviation, Phase 7 AUTH-02)" + - "DecryptLaravelPayload is cutover-import-only and is never called from Encrypted Scan/Value" + +patterns-established: + - "NewEncrypted(plaintext) for writes; Reveal() for reads; MarshalJSON/String/GoString always [redacted]" + - "One Register call per model/migration file init(); plugin.go is not edited" + - "Secret columns are json:\"-\" plus Hidden(); credential secrets are additionally lagoon.Encrypted" + +requirements-completed: [DATA-07, DATA-09] + +duration: 15min +completed: 2026-09-18 +--- + +# Phase 5 Plan 03: Secrets slice Summary + +**AES-256-GCM `lagoon.Encrypted` with stdlib HKDF and `app.previous_keys` rotation, `summer key:generate`, a standalone Laravel CBC decrypt helper, user-plugin `golem15_user_organisations`, and Fonoteka api_tokens/OAuth/credential tables** + +## Performance + +- **Duration:** 15 min +- **Started:** 2026-09-18T17:26:51Z +- **Completed:** 2026-09-18T17:41:45Z +- **Tasks:** 3 +- **Files modified:** 28 + +## Accomplishments + +- `lagoon.Encrypted` round-trips AES-256-GCM ciphertext keyed by HKDF from `app.key`, redacts on JSON/`String`/`%#v`, and decrypts rows after primary rotation via `app.previous_keys` +- Missing, short, or undecodable `app.key` fails loudly mentioning `SUMMER_APP__KEY`; `key:generate` prints a fresh 32-byte base64 key using `crypto/rand` +- `DecryptLaravelPayload` decrypts a real Laravel AES-256-CBC+HMAC fixture and is not on the live Scan/Value path +- `golem15_user_organisations` (slug/name/description, structure only) is created by the user plugin; no users-table ALTER ships this phase +- Three Fonoteka migrations create api_tokens, 3 OAuth tables, and 4 credential tables; hash/secret columns are `json:"-"`/`Hidden()`, and `api_key`/`token` are `lagoon.Encrypted` + +## Task Commits + +Each task was committed atomically: + +1. **Task 1 RED: Encrypted tests** - `06bad37` (test) in summercms.go +2. **Task 1 GREEN: Encrypted implementation** - `8d9109a` (feat) in summercms.go +3. **Task 2: create_organisations in user plugin** - `c291280` (feat) in fonoteka.go +4. **Task 3: secrets-slice migrations and models** - `f3bf0dd` (feat) in fonoteka.go + +**Plan metadata:** pending (this file) + +## Files Created/Modified + +- `lagoon/encrypted.go`, `lagoon/encrypted_test.go` — AES-256-GCM cast, HKDF, previous_keys, redaction +- `lagoon/keygen.go`, `lagoon/keygen_test.go` — `key:generate` +- `lagoon/laravel_decrypt.go`, `lagoon/laravel_decrypt_test.go` — cutover-only Laravel payload helper +- `lagoon/commands.go` — RuntimeCommands appends KeyGenerateCommand +- `lagoon/connection.go` — OpenFromApp publishes encryption keys from `app.key` +- `fonoteka.go/config/app.yaml` — documented empty `app.key` placeholder +- `fonoteka.go/plugins/golem15/user/updates/10_organisations.go`, `models/organisation.go` — FK target table +- `fonoteka.go/plugins/golem15/fonoteka/updates/11_secrets_slice.go` — api_tokens / OAuth / credentials +- `fonoteka.go/plugins/golem15/fonoteka/models/{api_token,oauth_*,*_credential}.go` — 8 models +- `fonoteka.go/parity/migrate_test.go` — 11 Fonoteka IDs, 2 user IDs, secrets rollback coverage + +## Decisions Made + +- Column keys use Go 1.27 `crypto/hkdf` (info `summercms.lagoon.encrypted.v1`); no new dependency +- Ciphertext format is version nibble `0x1` plus key-id, 12-byte nonce, GCM ciphertext+tag, base64 for `text` columns +- `OpenFromApp` is the boot wiring site for `LoadAppKey` + `PublishEncryptionKeys`; Scan/Value use the package-level key set +- Organisations belong to `golem15.user`; Phase 5 does not ALTER `users` (confirmed D-03 deviation, Phase 7 AUTH-02) +- `DecryptLaravelPayload` stays unwired from the live cast (D-10) + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 2 - Missing Critical] Write-path constructor for Encrypted** +- **Found during:** Task 1 (GREEN) +- **Issue:** Scan/Value/Reveal cannot set plaintext from another package +- **Fix:** Exported `NewEncrypted(plaintext string)` +- **Files modified:** `lagoon/encrypted.go` +- **Verification:** `TestEncryptedRoundTrip` +- **Committed in:** `8d9109a` + +**2. [Rule 3 - Blocking] OpenFromApp now requires a valid app.key** +- **Found during:** Task 1 (wiring PublishEncryptionKeys into OpenFromApp) +- **Issue:** `TestOpenFromAppReadsDSN` had no `SUMMER_APP__KEY` +- **Fix:** Test supplies a 32-byte base64 key via environ +- **Files modified:** `lagoon/connection_test.go` +- **Verification:** `go test ./lagoon/...` +- **Committed in:** `8d9109a` + +**3. [Rule 3 - Blocking] RuntimeCommands name list omitted key:generate** +- **Found during:** Task 1 +- **Issue:** `TestRuntimeCommandsRegisterBareAndColonNames` expected the old trio +- **Fix:** Assert `key:generate` is registered +- **Files modified:** `lagoon/migrations_test.go` +- **Verification:** that test +- **Committed in:** `8d9109a` + +**4. [Rule 2 - Missing Critical] Organisations columns from PHP** +- **Found during:** Task 2 (reading `create_organisations_table.php`) +- **Issue:** Plan listed id/name/timestamps as minimum; PHP also has unique `slug` and nullable `description` +- **Fix:** Table and model include slug and description; no membership/role tables +- **Files modified:** `plugins/golem15/user/updates/10_organisations.go`, `models/organisation.go` +- **Verification:** `TestCreateOrganisationsMigration` +- **Committed in:** `c291280` + +**5. [Rule 3 - Blocking] Existing migrate tests hardcoded ID counts** +- **Found during:** Task 3 +- **Issue:** Parity tests expected 1 user ID and 8 Fonoteka IDs; last rollback targeted album_reservations +- **Fix:** Expect 2 user IDs and 11 Fonoteka IDs; RollbackLast now drops credential tables first +- **Files modified:** `fonoteka.go/parity/migrate_test.go` +- **Verification:** `TestMigrateSeedsCanonicalGenres`, `TestSecretsSliceMigrationsUpDown`, `TestRollbackLastIsolatesFonoteka` +- **Committed in:** `f3bf0dd` + +**6. [Rule 3 - Blocking] Acceptance grep for widen_users vs required doc comment** +- **Found during:** Task 2 +- **Issue:** Plan required a comment naming the omitted users ALTER, which would fail `grep -rln widen_users` +- **Fix:** Comment states the decision without the `widen_users` token +- **Files modified:** `plugins/golem15/user/updates/10_organisations.go` +- **Verification:** grep returns no matches +- **Committed in:** `c291280` + +--- + +**Total deviations:** 6 auto-fixed (2 missing critical, 4 blocking) +**Impact on plan:** Required for a compiling write path, boot-time key loading, PHP column fidelity, and green existing migrate tests. No scope creep. `LoadAppKey` returns `[][]byte` previous keys rather than a concatenated blob so Scan can try each key. + +## Issues Encountered + +None beyond the auto-fixes above. + +## Authentication Gates + +None. + +## TDD Gate Compliance + +- RED: `06bad37` `test(05-03): add failing tests for Encrypted, key:generate, Laravel decrypt` +- GREEN: `8d9109a` `feat(05-03): implement lagoon.Encrypted, key:generate, Laravel decrypt helper` +- REFACTOR: not needed + +## Known Stubs + +- `fonoteka.go/config/app.yaml` `app.key: ""` is an intentional documented placeholder; empty fails boot once an encrypted-cast model runs / `OpenFromApp` loads keys. Operators set `SUMMER_APP__KEY` from `summer key:generate`. + +## User Setup Required + +None - no external service configuration required. Operators must set `SUMMER_APP__KEY` before serving or migrating through `OpenFromApp`. + +## Next Phase Readiness + +Ready for 05-04 (attachments). Encrypted credentials, OAuth tables, and the organisations FK target exist. zitadel/oidc storage remains Phase 8; organisation membership/roles remain Phase 7. + +## Self-Check: PASSED + +- Key files exist on disk (`lagoon/encrypted.go`, `keygen.go`, `laravel_decrypt.go`, user `10_organisations.go`, fonoteka `11_secrets_slice.go`, 8 models) +- Commits `06bad37`, `8d9109a` (summercms.go) and `c291280`, `f3bf0dd` (fonoteka.go) exist +- `go vet ./lagoon/...` and `go test ./lagoon/...` green in summercms.go +- `go vet ./...` and `go test ./...` green in fonoteka.go +- `DecryptLaravelPayload(` is called only from its file and test +- `lagoon/encrypted.go` does not import `golang.org/x/crypto`; `keygen.go` does not import `math/rand` + +--- +*Phase: 05-data-layer-full-fidelity* +*Completed: 2026-09-18*