215 lines
10 KiB
Markdown
215 lines
10 KiB
Markdown
---
|
|
phase: 05-data-layer-full-fidelity
|
|
plan: 03
|
|
subsystem: database
|
|
tags: [aes-gcm, hkdf, encrypted, gormigrate, oauth, credentials, organisations]
|
|
|
|
requires:
|
|
- phase: 05-data-layer-full-fidelity
|
|
provides: Winter-directory registries, lagoon.Fill, lagoon.Jsonable, HasHidden
|
|
provides:
|
|
- lagoon.Encrypted AES-256-GCM Scanner/Valuer with stdlib HKDF and previous_keys fallback
|
|
- summer key:generate and standalone DecryptLaravelPayload (unwired from Scan/Value)
|
|
- golem15_user_organisations structure-only table in the user plugin
|
|
- api_tokens, 3 OAuth tables, 4 credential tables and models with Hidden/json:"-"/Encrypted
|
|
affects: [05-04-attachments, 05-05-remaining-models, 05-06-tests, 07-user-plugin, 08-oauth]
|
|
|
|
tech-stack:
|
|
added: []
|
|
patterns:
|
|
- Encrypted plaintext is unexported; Reveal() is the only greppable accessor
|
|
- OpenFromApp loads app.key once and PublishEncryptionKeys so rows do not re-read config
|
|
- Organisations owned by golem15.user even though only Fonoteka credentials FK this phase
|
|
|
|
key-files:
|
|
created:
|
|
- lagoon/encrypted.go
|
|
- lagoon/encrypted_test.go
|
|
- lagoon/keygen.go
|
|
- lagoon/keygen_test.go
|
|
- lagoon/laravel_decrypt.go
|
|
- lagoon/laravel_decrypt_test.go
|
|
- plugins/golem15/user/updates/10_organisations.go
|
|
- plugins/golem15/user/models/organisation.go
|
|
- plugins/golem15/fonoteka/updates/11_secrets_slice.go
|
|
- plugins/golem15/fonoteka/models/api_token.go
|
|
- plugins/golem15/fonoteka/models/oauth_client.go
|
|
- plugins/golem15/fonoteka/models/oauth_auth_code.go
|
|
- plugins/golem15/fonoteka/models/oauth_refresh_token.go
|
|
- plugins/golem15/fonoteka/models/user_ai_credential.go
|
|
- plugins/golem15/fonoteka/models/org_ai_credential.go
|
|
- plugins/golem15/fonoteka/models/user_discogs_credential.go
|
|
- plugins/golem15/fonoteka/models/org_discogs_credential.go
|
|
modified:
|
|
- lagoon/commands.go
|
|
- lagoon/connection.go
|
|
- fonoteka.go/config/app.yaml
|
|
- fonoteka.go/parity/migrate_test.go
|
|
- fonoteka.go/plugins/golem15/user/go.mod
|
|
|
|
key-decisions:
|
|
- "Column keys are HKDF-SHA256 derived via Go 1.27 crypto/hkdf with info summercms.lagoon.encrypted.v1; no golang.org/x/crypto"
|
|
- "Ciphertext is 1-byte format/key-id | 12-byte nonce | GCM seal, stored base64 in text columns"
|
|
- "OpenFromApp calls LoadAppKey+PublishEncryptionKeys once per boot; empty/short/undecodable app.key fails with SUMMER_APP__KEY"
|
|
- "golem15_user_organisations is owned by the user plugin; Phase 5 ships no users-table ALTER (D-03 deviation, Phase 7 AUTH-02)"
|
|
- "DecryptLaravelPayload is cutover-import-only and is never called from Encrypted Scan/Value"
|
|
|
|
patterns-established:
|
|
- "NewEncrypted(plaintext) for writes; Reveal() for reads; MarshalJSON/String/GoString always [redacted]"
|
|
- "One Register call per model/migration file init(); plugin.go is not edited"
|
|
- "Secret columns are json:\"-\" plus Hidden(); credential secrets are additionally lagoon.Encrypted"
|
|
|
|
requirements-completed: [DATA-07, DATA-09]
|
|
|
|
duration: 15min
|
|
completed: 2026-09-18
|
|
---
|
|
|
|
# Phase 5 Plan 03: Secrets slice Summary
|
|
|
|
**AES-256-GCM `lagoon.Encrypted` with stdlib HKDF and `app.previous_keys` rotation, `summer key:generate`, a standalone Laravel CBC decrypt helper, user-plugin `golem15_user_organisations`, and Fonoteka api_tokens/OAuth/credential tables**
|
|
|
|
## Performance
|
|
|
|
- **Duration:** 15 min
|
|
- **Started:** 2026-09-18T17:26:51Z
|
|
- **Completed:** 2026-09-18T17:41:45Z
|
|
- **Tasks:** 3
|
|
- **Files modified:** 28
|
|
|
|
## Accomplishments
|
|
|
|
- `lagoon.Encrypted` round-trips AES-256-GCM ciphertext keyed by HKDF from `app.key`, redacts on JSON/`String`/`%#v`, and decrypts rows after primary rotation via `app.previous_keys`
|
|
- Missing, short, or undecodable `app.key` fails loudly mentioning `SUMMER_APP__KEY`; `key:generate` prints a fresh 32-byte base64 key using `crypto/rand`
|
|
- `DecryptLaravelPayload` decrypts a real Laravel AES-256-CBC+HMAC fixture and is not on the live Scan/Value path
|
|
- `golem15_user_organisations` (slug/name/description, structure only) is created by the user plugin; no users-table ALTER ships this phase
|
|
- Three Fonoteka migrations create api_tokens, 3 OAuth tables, and 4 credential tables; hash/secret columns are `json:"-"`/`Hidden()`, and `api_key`/`token` are `lagoon.Encrypted`
|
|
|
|
## Task Commits
|
|
|
|
Each task was committed atomically:
|
|
|
|
1. **Task 1 RED: Encrypted tests** - `06bad37` (test) in summercms.go
|
|
2. **Task 1 GREEN: Encrypted implementation** - `8d9109a` (feat) in summercms.go
|
|
3. **Task 2: create_organisations in user plugin** - `c291280` (feat) in fonoteka.go
|
|
4. **Task 3: secrets-slice migrations and models** - `f3bf0dd` (feat) in fonoteka.go
|
|
|
|
**Plan metadata:** pending (this file)
|
|
|
|
## Files Created/Modified
|
|
|
|
- `lagoon/encrypted.go`, `lagoon/encrypted_test.go` — AES-256-GCM cast, HKDF, previous_keys, redaction
|
|
- `lagoon/keygen.go`, `lagoon/keygen_test.go` — `key:generate`
|
|
- `lagoon/laravel_decrypt.go`, `lagoon/laravel_decrypt_test.go` — cutover-only Laravel payload helper
|
|
- `lagoon/commands.go` — RuntimeCommands appends KeyGenerateCommand
|
|
- `lagoon/connection.go` — OpenFromApp publishes encryption keys from `app.key`
|
|
- `fonoteka.go/config/app.yaml` — documented empty `app.key` placeholder
|
|
- `fonoteka.go/plugins/golem15/user/updates/10_organisations.go`, `models/organisation.go` — FK target table
|
|
- `fonoteka.go/plugins/golem15/fonoteka/updates/11_secrets_slice.go` — api_tokens / OAuth / credentials
|
|
- `fonoteka.go/plugins/golem15/fonoteka/models/{api_token,oauth_*,*_credential}.go` — 8 models
|
|
- `fonoteka.go/parity/migrate_test.go` — 11 Fonoteka IDs, 2 user IDs, secrets rollback coverage
|
|
|
|
## Decisions Made
|
|
|
|
- Column keys use Go 1.27 `crypto/hkdf` (info `summercms.lagoon.encrypted.v1`); no new dependency
|
|
- Ciphertext format is version nibble `0x1` plus key-id, 12-byte nonce, GCM ciphertext+tag, base64 for `text` columns
|
|
- `OpenFromApp` is the boot wiring site for `LoadAppKey` + `PublishEncryptionKeys`; Scan/Value use the package-level key set
|
|
- Organisations belong to `golem15.user`; Phase 5 does not ALTER `users` (confirmed D-03 deviation, Phase 7 AUTH-02)
|
|
- `DecryptLaravelPayload` stays unwired from the live cast (D-10)
|
|
|
|
## Deviations from Plan
|
|
|
|
### Auto-fixed Issues
|
|
|
|
**1. [Rule 2 - Missing Critical] Write-path constructor for Encrypted**
|
|
- **Found during:** Task 1 (GREEN)
|
|
- **Issue:** Scan/Value/Reveal cannot set plaintext from another package
|
|
- **Fix:** Exported `NewEncrypted(plaintext string)`
|
|
- **Files modified:** `lagoon/encrypted.go`
|
|
- **Verification:** `TestEncryptedRoundTrip`
|
|
- **Committed in:** `8d9109a`
|
|
|
|
**2. [Rule 3 - Blocking] OpenFromApp now requires a valid app.key**
|
|
- **Found during:** Task 1 (wiring PublishEncryptionKeys into OpenFromApp)
|
|
- **Issue:** `TestOpenFromAppReadsDSN` had no `SUMMER_APP__KEY`
|
|
- **Fix:** Test supplies a 32-byte base64 key via environ
|
|
- **Files modified:** `lagoon/connection_test.go`
|
|
- **Verification:** `go test ./lagoon/...`
|
|
- **Committed in:** `8d9109a`
|
|
|
|
**3. [Rule 3 - Blocking] RuntimeCommands name list omitted key:generate**
|
|
- **Found during:** Task 1
|
|
- **Issue:** `TestRuntimeCommandsRegisterBareAndColonNames` expected the old trio
|
|
- **Fix:** Assert `key:generate` is registered
|
|
- **Files modified:** `lagoon/migrations_test.go`
|
|
- **Verification:** that test
|
|
- **Committed in:** `8d9109a`
|
|
|
|
**4. [Rule 2 - Missing Critical] Organisations columns from PHP**
|
|
- **Found during:** Task 2 (reading `create_organisations_table.php`)
|
|
- **Issue:** Plan listed id/name/timestamps as minimum; PHP also has unique `slug` and nullable `description`
|
|
- **Fix:** Table and model include slug and description; no membership/role tables
|
|
- **Files modified:** `plugins/golem15/user/updates/10_organisations.go`, `models/organisation.go`
|
|
- **Verification:** `TestCreateOrganisationsMigration`
|
|
- **Committed in:** `c291280`
|
|
|
|
**5. [Rule 3 - Blocking] Existing migrate tests hardcoded ID counts**
|
|
- **Found during:** Task 3
|
|
- **Issue:** Parity tests expected 1 user ID and 8 Fonoteka IDs; last rollback targeted album_reservations
|
|
- **Fix:** Expect 2 user IDs and 11 Fonoteka IDs; RollbackLast now drops credential tables first
|
|
- **Files modified:** `fonoteka.go/parity/migrate_test.go`
|
|
- **Verification:** `TestMigrateSeedsCanonicalGenres`, `TestSecretsSliceMigrationsUpDown`, `TestRollbackLastIsolatesFonoteka`
|
|
- **Committed in:** `f3bf0dd`
|
|
|
|
**6. [Rule 3 - Blocking] Acceptance grep for widen_users vs required doc comment**
|
|
- **Found during:** Task 2
|
|
- **Issue:** Plan required a comment naming the omitted users ALTER, which would fail `grep -rln widen_users`
|
|
- **Fix:** Comment states the decision without the `widen_users` token
|
|
- **Files modified:** `plugins/golem15/user/updates/10_organisations.go`
|
|
- **Verification:** grep returns no matches
|
|
- **Committed in:** `c291280`
|
|
|
|
---
|
|
|
|
**Total deviations:** 6 auto-fixed (2 missing critical, 4 blocking)
|
|
**Impact on plan:** Required for a compiling write path, boot-time key loading, PHP column fidelity, and green existing migrate tests. No scope creep. `LoadAppKey` returns `[][]byte` previous keys rather than a concatenated blob so Scan can try each key.
|
|
|
|
## Issues Encountered
|
|
|
|
None beyond the auto-fixes above.
|
|
|
|
## Authentication Gates
|
|
|
|
None.
|
|
|
|
## TDD Gate Compliance
|
|
|
|
- RED: `06bad37` `test(05-03): add failing tests for Encrypted, key:generate, Laravel decrypt`
|
|
- GREEN: `8d9109a` `feat(05-03): implement lagoon.Encrypted, key:generate, Laravel decrypt helper`
|
|
- REFACTOR: not needed
|
|
|
|
## Known Stubs
|
|
|
|
- `fonoteka.go/config/app.yaml` `app.key: ""` is an intentional documented placeholder; empty fails boot once an encrypted-cast model runs / `OpenFromApp` loads keys. Operators set `SUMMER_APP__KEY` from `summer key:generate`.
|
|
|
|
## User Setup Required
|
|
|
|
None - no external service configuration required. Operators must set `SUMMER_APP__KEY` before serving or migrating through `OpenFromApp`.
|
|
|
|
## Next Phase Readiness
|
|
|
|
Ready for 05-04 (attachments). Encrypted credentials, OAuth tables, and the organisations FK target exist. zitadel/oidc storage remains Phase 8; organisation membership/roles remain Phase 7.
|
|
|
|
## Self-Check: PASSED
|
|
|
|
- Key files exist on disk (`lagoon/encrypted.go`, `keygen.go`, `laravel_decrypt.go`, user `10_organisations.go`, fonoteka `11_secrets_slice.go`, 8 models)
|
|
- Commits `06bad37`, `8d9109a` (summercms.go) and `c291280`, `f3bf0dd` (fonoteka.go) exist
|
|
- `go vet ./lagoon/...` and `go test ./lagoon/...` green in summercms.go
|
|
- `go vet ./...` and `go test ./...` green in fonoteka.go
|
|
- `DecryptLaravelPayload(` is called only from its file and test
|
|
- `lagoon/encrypted.go` does not import `golang.org/x/crypto`; `keygen.go` does not import `math/rand`
|
|
|
|
---
|
|
*Phase: 05-data-layer-full-fidelity*
|
|
*Completed: 2026-09-18*
|