196 lines
19 KiB
Markdown
196 lines
19 KiB
Markdown
---
|
|
phase: 05-data-layer-full-fidelity
|
|
plan: 06
|
|
type: execute
|
|
wave: 5
|
|
depends_on: ["05-05"]
|
|
files_modified:
|
|
- fonoteka.go/plugins/golem15/fonoteka/classes/album_write_service_fuzz_test.go
|
|
- fonoteka.go/plugins/golem15/fonoteka/classes/collection_write_service_fuzz_test.go
|
|
- fonoteka.go/plugins/golem15/fonoteka/classes/credential_write_service_fuzz_test.go
|
|
- summercms.go/lagoon/fill_fuzz_test.go
|
|
- summercms.go/lagoon/hidden_marshal_test.go
|
|
- fonoteka.go/plugins/golem15/fonoteka/classes/collection_public_token_test.go
|
|
- fonoteka.go/plugins/golem15/fonoteka/classes/attach_smoke_test.go
|
|
- .planning/phases/05-data-layer-full-fidelity/05-SECURITY-REVIEW.md
|
|
autonomous: true
|
|
requirements: [DATA-06, DATA-07, DATA-09]
|
|
|
|
must_haves:
|
|
truths:
|
|
- "lagoon.Fill has its own framework-level fuzz test on a fixture model, asserting random extra/unknown keys are never set on the struct (D-05, D-06, D-07)"
|
|
- "AlbumWriteService, CollectionWriteService and all four credential write paths are fuzzed against real Postgres with random extra and server-owned keys, asserting nothing outside each service's fill boundary is ever persisted (D-07)"
|
|
- "A test marshals every model registered in models.All() (both plugins) and asserts every name in that model's Hidden() is absent from the JSON output (D-08)"
|
|
- "Soft-deleting then attempting to recreate a Collection with the same public_token reproduces PHP's actual plain-UNIQUE-constraint behavior, not a partial-index improvement (success criterion 5, RESEARCH.md soft-delete+unique audit)"
|
|
- "A security-review pass covers the fill and encryption boundaries specifically, with each finding mapped to a passing test or an accepted/documented risk"
|
|
artifacts:
|
|
- path: .planning/phases/05-data-layer-full-fidelity/05-SECURITY-REVIEW.md
|
|
provides: "Security review pass mapping every T-05-xx threat across all 5 prior plans to a passing test or an explicit accept/transfer rationale"
|
|
key_links:
|
|
- from: fonoteka.go/plugins/golem15/fonoteka/classes/album_write_service_fuzz_test.go
|
|
to: fonoteka.go/plugins/golem15/fonoteka/classes/album_write_service.go
|
|
via: "fuzz target calls SaveAlbum with random extra/server-owned keys"
|
|
pattern: "func FuzzSaveAlbum"
|
|
- from: summercms.go/lagoon/hidden_marshal_test.go
|
|
to: fonoteka.go/plugins/golem15/fonoteka/models/registry.go
|
|
via: "test imports both plugins' models.All() and marshals every registered type"
|
|
pattern: "models\\.All\\(\\)|user/models\\.All\\(\\)"
|
|
---
|
|
|
|
<objective>
|
|
Close the phase with full unit/fuzz coverage of everything Plans 05-01 through 05-05 built, plus the security-review pass CONTEXT.md names as mandatory for this phase's mass-assignment and encryption boundaries. This is the last plan of the phase per the project's lean-mode workflow rule — earlier plans carried smoke tests but were never blocked on full coverage; this plan is what closes that gap.
|
|
|
|
Purpose: prove, not just assert, that D-05/D-06/D-07's two-layer fillable boundary actually rejects unknown/server-owned keys under fuzzing against real Postgres, that D-08's hidden discipline holds across every one of the 25+ registered models (not just the ones spot-checked while writing them), and that the one real soft-delete+unique interaction behaves exactly like PHP's.
|
|
Output: fuzz tests for `lagoon.Fill` and every fill-boundary write service named in D-07 (Album, Collection, the 4 credential models); a hidden-marshal test over every registered model in both plugins; the delete-then-recreate `public_token` behavioral test; `05-SECURITY-REVIEW.md`.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
|
@$HOME/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/phases/05-data-layer-full-fidelity/05-CONTEXT.md
|
|
@.planning/phases/05-data-layer-full-fidelity/05-RESEARCH.md
|
|
@.planning/phases/05-data-layer-full-fidelity/05-01-SUMMARY.md
|
|
@.planning/phases/05-data-layer-full-fidelity/05-02-SUMMARY.md
|
|
@.planning/phases/05-data-layer-full-fidelity/05-03-SUMMARY.md
|
|
@.planning/phases/05-data-layer-full-fidelity/05-04-SUMMARY.md
|
|
@.planning/phases/05-data-layer-full-fidelity/05-05-SUMMARY.md
|
|
</context>
|
|
|
|
<interfaces>
|
|
<!-- Contracts every prior plan in this phase shipped, consumed here for verification only -->
|
|
|
|
```go
|
|
// lagoon (Plan 05-01/05-03)
|
|
func Fill(model any, allowed []string, requested map[string]any, production bool) error
|
|
type HasHidden interface { Hidden() []string }
|
|
type Encrypted struct{ /* unexported */ }
|
|
func (e Encrypted) Reveal() string
|
|
|
|
// fonoteka plugin classes (Plan 05-02)
|
|
var AlbumFillFields []string
|
|
func SaveAlbum(ctx context.Context, gdb *gorm.DB, album *models.Album, requested map[string]any, artistIDs []uint) error
|
|
var CollectionFillFields []string
|
|
func SaveCollection(ctx context.Context, gdb *gorm.DB, collection *models.Collection, requested map[string]any) error
|
|
|
|
// models registries, both plugins (Plan 05-01)
|
|
func All() []any
|
|
```
|
|
</interfaces>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 1 (summercms.go, fonoteka.go): Fill boundary fuzz — lagoon.Fill framework fuzz + Album/Collection/4-credential-model service-level fuzz against real Postgres</name>
|
|
<files>summercms.go/lagoon/fill_fuzz_test.go, fonoteka.go/plugins/golem15/fonoteka/classes/album_write_service_fuzz_test.go, fonoteka.go/plugins/golem15/fonoteka/classes/collection_write_service_fuzz_test.go, fonoteka.go/plugins/golem15/fonoteka/classes/credential_write_service_fuzz_test.go</files>
|
|
<read_first>
|
|
summercms.go/lagoon/fill.go (Plan 05-01)
|
|
fonoteka.go/plugins/golem15/fonoteka/classes/album_write_service.go, collection_write_service.go (Plan 05-02)
|
|
fonoteka.go/plugins/golem15/fonoteka/models/{user_ai_credential,org_ai_credential,user_discogs_credential,org_discogs_credential}.go (Plan 05-03)
|
|
.planning/phases/05-data-layer-full-fidelity/05-CONTEXT.md (D-07's exact wording: "fuzzed against real Postgres with random extra and server-owned keys, asserting nothing outside the list is persisted")
|
|
fonoteka.go/parity/migrate_test.go (activateAppPlugins/parityDB harness this fuzz test reuses)
|
|
</read_first>
|
|
<action>
|
|
In `lagoon/fill_fuzz_test.go`: add `func FuzzFill(f *testing.F)` seeding with a handful of known-shape maps (some keys allowed, some not, some `nil` values, some non-string keys coerced) against a small fixture struct declared in the test file; the fuzz body asserts the fixture's non-allow-listed fields are always left at their zero value after `Fill` runs, regardless of what the fuzzer threw at the `requested` map, and that `Fill` never panics on any input (including deeply nested/weird `any` values, empty maps, and a `nil` map).
|
|
|
|
In `classes/album_write_service_fuzz_test.go`: add `func FuzzSaveAlbum(f *testing.F)` — for each fuzzed input (a `map[string]any` including random keys drawn from a mix of real column names outside `AlbumFillFields` — specifically always include `collection_id` and `market_price_source` as adversarial server-owned keys per D-07's explicit callout — plus fuzzer-generated garbage keys), call `SaveAlbum` against the real testcontainers Postgres (reuse `activateAppPlugins`/`parityDB`), then re-load the saved row directly via `SELECT` and assert `collection_id` and `market_price_source` still hold their pre-fuzz values, never the fuzzed input's values, and that no fuzzer-generated garbage key ever appears as a real column write (this last assertion is necessarily structural: assert the row's *actual* fillable-column values match only what `AlbumFillFields` allowed through, by comparing against a parallel in-memory application of `Fill` with the same inputs). Mirror the same shape in `collection_write_service_fuzz_test.go` for `SaveCollection`/`CollectionFillFields`.
|
|
|
|
In `classes/credential_write_service_fuzz_test.go`: for each of the 4 credential models (`UserAiCredential`, `OrgAiCredential`, `UserDiscogsCredential`, `OrgDiscogsCredential`), fuzz a save path using each model's own `Fillable()` as the allow-list (per D-07, "at minimum Album, Collection and the four credential models" — these four have no dedicated write-service file per Plan 05-03, so fuzz `lagoon.Fill` + a direct `tx.Save` call for each, asserting the `Encrypted`-typed `api_key`/`token` column round-trips correctly under fuzzed non-credential keys and that no adversarial key (e.g. a fuzzed `organisation_id`/`user_id` override attempt) ever changes the row's owner FK away from what the test explicitly set before calling `Fill`).
|
|
</action>
|
|
<verify>
|
|
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go test ./lagoon/... -fuzz=FuzzFill -fuzztime=20s && cd /media/nvme/dev/golem15/summercms.io/summercms/fonoteka.go && go test ./plugins/golem15/fonoteka/classes/... -fuzz=FuzzSaveAlbum -fuzztime=20s</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- `FuzzFill` runs for at least 20s with zero crashers/panics and zero corpus entries where a non-allowed field was set.
|
|
- `FuzzSaveAlbum`/`FuzzSaveCollection` each run for at least 20s against real Postgres with zero crashers, and zero cases where `collection_id`/`market_price_source` (or Collection's excluded fields) changed from their pre-fuzz value.
|
|
- The 4 credential fuzz tests each assert the row's FK owner column is never altered by a fuzzed key.
|
|
</acceptance_criteria>
|
|
<done>`lagoon.Fill` and all 6 named write paths (Album, Collection, 4 credential models) are fuzz-tested against real Postgres with zero found violations of the fillable boundary.</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 2 (summercms.go, fonoteka.go): Hidden-marshal test over every registered model; Collection.public_token delete-then-recreate behavior</name>
|
|
<files>summercms.go/lagoon/hidden_marshal_test.go, fonoteka.go/plugins/golem15/fonoteka/classes/collection_public_token_test.go</files>
|
|
<read_first>
|
|
summercms.go/lagoon/fill.go (HasHidden)
|
|
fonoteka.go/plugins/golem15/fonoteka/models/registry.go, fonoteka.go/plugins/golem15/user/models/registry.go (Plan 05-01, both All() functions)
|
|
.planning/phases/05-data-layer-full-fidelity/05-RESEARCH.md ("Soft-delete + unique audit" section — golem15_fonoteka_collections.public_token is the one real target, plain UNIQUE, no partial index)
|
|
</read_first>
|
|
<action>
|
|
In `lagoon/hidden_marshal_test.go` (placed in `lagoon` so it can import both plugin modules as test-only dependencies, or in a small dedicated test package under `fonoteka.go/parity` if importing both plugins from `summercms.go`'s test tree is not viable given the module boundary — pick whichever avoids a circular/awkward import and document the choice in a comment): iterate every value in `fonoteka.go/plugins/golem15/fonoteka/models.All()` and `fonoteka.go/plugins/golem15/user/models.All()`, type-assert each against `lagoon.HasHidden`, and for every one that implements it, populate a zero-value instance's hidden fields with a distinctive sentinel string (via reflection, matching the field to its `gorm:"column:<name>"` tag), `json.Marshal` it, and assert the sentinel string is never present in the output. Also assert every field carrying a `json:"-"` tag is likewise absent, as an independent structural check (parse the struct's own field tags via `reflect.Type`, not just `Hidden()`'s stated list) — this catches a `Hidden()` implementation drifting out of sync with its own `json:"-"` tags.
|
|
|
|
In `classes/collection_public_token_test.go`: soft-delete a `Collection` row that has a `PublicToken` set, then attempt to create a second `Collection` with the identical literal `public_token` value, and assert this reproduces PHP's actual behavior — a real Postgres unique-constraint violation (the plain `UNIQUE` constraint applies regardless of `deleted_at`, per RESEARCH.md's audit) — not a fabricated "success" that a partial index would have allowed. Also test the inverse: after the first Collection is *hard*-deleted (`Unscoped().Delete`), the same `public_token` value is now free to reuse.
|
|
</action>
|
|
<verify>
|
|
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go test ./lagoon/... -run TestHiddenNeverMarshals && cd /media/nvme/dev/golem15/summercms.io/summercms/fonoteka.go && go test ./plugins/golem15/fonoteka/classes/... -run TestCollectionPublicTokenDeleteThenRecreate</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- The hidden-marshal test enumerates and checks every model in both `models.All()` slices (assert the enumerated count matches the expected 25+ model total, so silently skipping a model — e.g. one that fails the `HasHidden` type assertion by mistake — is itself a caught regression).
|
|
- `TestCollectionPublicTokenDeleteThenRecreate` asserts a real constraint-violation error is returned after a soft-delete, and asserts success after a hard-delete of the same row.
|
|
</acceptance_criteria>
|
|
<done>Every registered model's hidden columns are provably unmarshalable, and the one real soft-delete+unique interaction matches PHP's actual (not "improved") behavior.</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 3 (fonoteka.go, .planning): Attachment lifecycle smoke test; security-review pass over the fill and encryption boundaries</name>
|
|
<files>fonoteka.go/plugins/golem15/fonoteka/classes/attach_smoke_test.go, .planning/phases/05-data-layer-full-fidelity/05-SECURITY-REVIEW.md</files>
|
|
<read_first>
|
|
summercms.go/lagoon/attach/file.go, thumb.go (Plan 05-04)
|
|
fonoteka.go/plugins/golem15/fonoteka/models/album.go, collection.go (MorphName, Plan 05-04)
|
|
.planning/phases/05-data-layer-full-fidelity/05-01-PLAN.md through 05-05-PLAN.md's `<threat_model>` blocks (every T-05-xx entry across all 5 prior plans)
|
|
</read_first>
|
|
<action>
|
|
Add an `attach_smoke_test.go` covering Collection's ordered `photos` and single `image` end to end against a `memblob` bucket: create a Collection, attach 3 photo `File` rows with distinct `sort_order`, attach 1 image `File` row, read them back ordered, call `.Thumb(200,200,"crop")` on the image and assert the returned URL contains the exact `thumb_<id>_200_200_0_0_crop.<ext>` filename, soft-delete the Collection and assert the files/blobs survive, then force-delete and assert the rows are gone and (per Plan 05-04's documented two-phase contract) the blobs are removed after the transaction commits.
|
|
|
|
Write `.planning/phases/05-data-layer-full-fidelity/05-SECURITY-REVIEW.md`: list every `T-05-*` threat ID from all 5 prior plans' `<threat_model>` blocks in one table, and for each, name the specific test (from this plan or an earlier one) that proves the mitigation holds, or restate the accept/transfer rationale verbatim from its originating plan. Focus the narrative specifically on the fill and encryption boundaries per CONTEXT.md's phase-boundary text ("apply the security-review agent... security review can grep" for `Reveal` calls): explicitly grep the full `fonoteka.go` and `summercms.go` trees for `.Reveal()` call sites and list every one found, confirming none appear in a logging statement, a `Serialize*` function, or any path reachable from marshal/String/GoString; explicitly grep for `Association(` to confirm no pivot write ever uses Association Mode; explicitly grep for `AutoMigrate` to confirm the zero-tolerance rule held across all 5 plans' new code.
|
|
</action>
|
|
<verify>
|
|
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/fonoteka.go && go test ./plugins/golem15/fonoteka/classes/... -run TestAttachSmoke && grep -rn "AutoMigrate" plugins/ | wc -l</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- `TestAttachSmoke` passes, exercising the full photo/image/thumb/soft-delete/force-delete lifecycle against `memblob`.
|
|
- `05-SECURITY-REVIEW.md` maps every `T-05-01` through the highest-numbered threat ID across all 5 prior plans to a named passing test or a restated accept/transfer rationale — none left unmapped.
|
|
- `grep -rn "AutoMigrate" fonoteka.go/plugins summercms.go/lagoon` returns zero matches.
|
|
- `grep -rln "\.Reveal()" fonoteka.go summercms.go` results are all enumerated by name in `05-SECURITY-REVIEW.md`, each confirmed not reachable from a logging or serialization path.
|
|
</acceptance_criteria>
|
|
<done>The attachment lifecycle is smoke-tested end to end; every threat named across the phase's 5 prior plans is mapped to a passing test or a restated, deliberate risk acceptance in a committed security review document.</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<threat_model>
|
|
## Trust Boundaries
|
|
|
|
| Boundary | Description |
|
|
|----------|--------------|
|
|
| this plan's tests -> every earlier plan's security-load-bearing code | this is the phase's closing verification pass, not new production code — its own risk surface is limited to test-code correctness |
|
|
|
|
## STRIDE Threat Register
|
|
|
|
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
|
|-----------|----------|-----------|-------------|-----------------|
|
|
| T-05-20 | Tampering | Fuzz corpus could under-explore the input space and miss a real violation | accept | 20s minimum fuzz time per target plus explicit adversarial seed corpus entries (collection_id, market_price_source, organisation_id overrides) named directly in the task rather than left to chance discovery |
|
|
| T-05-21 | Information Disclosure | The hidden-marshal test itself could silently skip a model if a future model forgets to implement HasHidden | mitigate | The test asserts the enumerated model count against the expected total, turning a silently-skipped model into a failing test rather than a false pass |
|
|
| T-05-22 | Repudiation | Security review completeness | mitigate | `05-SECURITY-REVIEW.md` is required to map every T-05-xx ID from all 5 prior plans by name — an unmapped ID is a review gap, not an accepted risk, and must be caught before phase close |
|
|
</threat_model>
|
|
|
|
<verification>
|
|
Full suite in both repos: `cd summercms.go && go vet ./... && go test ./...` (including `-fuzz` targets for at least 20s each) and `cd ../fonoteka.go && go vet ./... && go test ./...` (testcontainers Postgres). `05-SECURITY-REVIEW.md` committed and cross-checked against every prior plan's `<threat_model>` block.
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- `lagoon.Fill` and all 6 D-07-named write paths are fuzz-tested against real Postgres with zero found fillable-boundary violations.
|
|
- Every registered model (25+ across both plugins) passes the hidden-marshal check.
|
|
- The one real soft-delete+unique interaction (`Collection.public_token`) matches PHP's actual constraint behavior.
|
|
- The attachment lifecycle is smoke-tested end to end.
|
|
- `05-SECURITY-REVIEW.md` maps every threat ID from all 5 prior plans to a passing test or a restated acceptance rationale.
|
|
- `go vet ./...` and `go test ./...` are green in both repos.
|
|
</success_criteria>
|
|
|
|
<output>
|
|
Create `.planning/phases/05-data-layer-full-fidelity/05-06-SUMMARY.md` when done
|
|
</output>
|