947aabee93
- new note .planning/notes/core-plugins-own-repos.md: shared core plugins live in sm-<name>-plugin repos mounted as submodules - 01-CONTEXT deferral points to the note; PROJECT constraint and Key Decisions row - ROADMAP Phase 12 repos and the 12-01 entry, and Phase 12 plans 12-01, 12-02, 12-05 name sm-user-plugin and the submodule commit workflow
48 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, estimate, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | estimate | must_haves | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 12-p-ytarium-api-collections-and-albums | 01 | execute | 1 |
|
true |
|
|
|
Phase Goal
ROADMAP Phase 12 goal (verbatim, not in user-story form; MVP precedent of Phases 11 and 11.2 is to quote it): Collections and Albums endpoints are ported with byte-compatible request/response shapes, including active-context switching, editor invitations, ratings, reservations, cover handling and search.
This plan's slice: the framework can produce every Laravel 422 body, record and replay multipart uploads, emit Winter-shaped upload URLs, decode webp, report the search engine's found count and rank fields by weight; the user plugin knows user groups; and the roadmap and requirements say what Phase 12 actually ships. Nothing here is user-visible on its own: plans 12-02 to 12-04 build every endpoint on these pieces.
Purpose: every Phase 12 endpoint needs Laravel-exact 422 bodies (D-21), multipart recordings (D-11), PHP upload URLs (D-22), webp (D-24), a re-gated search total (D-19) and the site-admin predicate (D-25).
Output: lagoon.ValidateRequest with pl/en catalogs; attach URL/PublicURL and webp; tide multipart and upload/publication masks; beachcomber PageSearcher and weights; user groups; README and docs updates; reworded ROADMAP/REQUIREMENTS.
Repos: summercms.go (framework and planning docs), sm-user-plugin (user groups, committed inside the submodule and pushed to its origin master first) and fonoteka.go (parity schema allow-list plus the bumped submodule pointer). Framework code, tests, READMEs and docs use neutral names (acme, blog, posts) and never name the application. Planning docs and code go in separate commits; never add co-author tags.
<execution_context>
@/.claude/gsd-core/workflows/execute-plan.md
@/.claude/gsd-core/templates/summary.md
</execution_context>
Artifacts this phase produces
(This plan's share.)
- lagoon:
ValidateRequest(ctx context.Context, tx *gorm.DB, input map[string]any, rules []RequestRule, tr *phrasebook.Translator) (map[string][]string, error),RequestRule{Field string; Rules []Rule},Rule(a parsed token or a custom func),ParseRules(spec string) []Rule,CustomRule(fn func(attribute string, value any) (message string, failed bool)) Rule,UploadedFile{Filename string; Size int64; Header textproto.MIMEHeader; Open func() (io.ReadCloser, error)}withUploadedFileFromHeader(*multipart.FileHeader) UploadedFile,ErrorKeys(errs map[string][]string) []string(rule-declaration order); catalog namespacelagoon::validation.*(pl, en). - attach:
PublicURL(key string) string,(*File).URL() string, webp decoding. - tide:
Request.Parts []Part,Part{Name, Value, File, Filename, ContentType, SHA256},MultipartBoundary(fixed), upload-URL masking forurl/thumb_url, album-date masking inNormalizePublications. - beachcomber:
PageSearcher{SearchPage(ctx, index string, q Query) (SearchResult, error)},SearchResult{IDs []string; Found int},SearchPage(ctx, e Engine, index string, q Query) (SearchResult, error)(fallback helper),Query.QueryByWeights []int. - User plugin (fonoteka.go): tables
user_groups,users_groups;models.UserGroup;models.User.Groups;classes.UserGroupCodes(ctx, db, userID) ([]string, error),classes.HasGroupCode(ctx, db, userID, code) (bool, error). - Dependency:
golang.org/x/image v0.46.0(direct, summercms.go).
Assumption-delta decision
<assumption_delta_decision>
Detector run on the Phase 12 ROADMAP section: detected=false. Considered D-25 (user groups) by hand: it adds a membership relation that a single predicate (site admin = group code admin) reads; the user identity, its primary key and the JWT principal stay the only identity model. Noun primary: User. Decision: no-change. Rationale: groups are an attribute of a user, not a second identity or tenant.
</assumption_delta_decision>
Flagged assumptions (edge probe)
- API-01 adjacency/empty/ordering and API-02 concurrency are not framework concerns; plans 12-02 to 12-04 resolve them. This plan resolves API-01 encoding (character counting) and API-02 boundary and precision for the validator.
(1) Catalogs: create modules/phrasebook/lang/pl/validation.yaml and en/validation.yaml as faithful ports of Winter's system lang validation.php files (every key, nested size maps between, gt, gte, lt, lte, max, min, size with numeric, file, string, array children, plus custom and attributes maps if present). Text is copied verbatim; only PHP quoting is converted. They load as lagoon::validation.* beside the untouched lagoon::validate.* group. Keys missing from pl (after_or_equal, before_or_equal and any other) are left missing so the translator falls back to en, as Winter does.
(2) validate_request.go and validate_rules.go: exported RequestRule{Field string; Rules []Rule}, Rule (token name, args, implicit flag, or a custom func), ParseRules(spec string) []Rule (pipe split, regex: arguments kept whole even when they contain a pipe or comma, in: args split on commas), CustomRule(fn) (non-implicit, runs in declaration position, its returned message is used verbatim), UploadedFile with UploadedFileFromHeader, and ValidateRequest(ctx, tx, input, rules, tr). Semantics to port from Validator.php: expand * segments against the decoded input (each array index becomes .0, .1 ...; an absent parent expands to the literal attribute so required still fires); per attribute run rules in order; a rule is validatable only when the value is present or the rule is implicit (required, required_* family, accepted, present), a blank string after trim counts as absent for non-implicit rules, nullable with a null value skips the rest, sometimes skips an absent key; stop the attribute after a failed implicit rule (shouldStopValidating) and after bail. Rules: required, nullable, sometimes, bail, array, string, integer (int types, json.Number without fraction, numeric strings without fraction as filter_var FILTER_VALIDATE_INT), numeric, boolean (true, false, 0, 1, "0", "1"), email (port FILTER_VALIDATE_EMAIL-compatible check used by Laravel's default email rule, record the boundary cases you choose in the test), url (Laravel 9 validateUrl regex), date (strtotime-compatible: accept the ISO and Y-m-d shapes the recorded fixtures use; document the accepted set), after_or_equal and before_or_equal (argument is a date or a relative word: today, tomorrow, yesterday, now), exists:table,column (identName-checked identifiers, deleted_at IS NULL is NOT added because Laravel's exists does not add it), regex:/pattern/ (PCRE delimiters stripped; reject patterns Go RE2 cannot compile with a boot-time error, never at request time), in, mimes and image (sniff with http.DetectContentType plus extension mapping as Laravel's guessExtension; image = jpg, jpeg, png, gif, bmp, svg, webp), min, max, between, size, same as Laravel getSize: string length in Unicode code points (utf8.RuneCountInString, PHP mb_strlen), array element count, numeric value when the attribute also has numeric or integer, file size in kilobytes (bytes/1024). Messages: key lagoon::validation.<rule>, or lagoon::validation.<rule>.<numeric|file|string|array> for size rules picked by the same type order Laravel uses; replacements :attribute (Laravel getDisplayableAttribute: custom attribute name if the catalog defines one, else snake case with underscores turned into spaces; wildcard attributes keep their dotted index), :min, :max, :size, :values (comma-joined), :date, :other, :format. Return the Laravel errors object (attribute to ordered messages) plus ErrorKeys giving attribute order by first rule failure so callers can emit keys in PHP order where order is ever compared byte-wise.
(3) Fold the min-message todo in validate.go: pick the message by which bound failed (min below the lower bound, max above the upper, between when both bounds came from between) for integer and numeric fields; keep the existing lagoon::validate.* texts and every other branch byte-identical. Before changing, grep the user-api fixtures for may not be greater than and must be at least and keep any recorded text green.
(4) Smoke tests (full coverage comes in 12-05): validate_request_test.go with neutral posts/tags examples including {"posts":[]} under required|array|min:1 producing exactly Pole posts jest wymagane. in pl and The posts field is required. in en; posts.*.title wildcard naming posts.0.title; a 255 versus 256 character Polish string under max:255; between:1889,2100 at 1888, 1889, 2100, 2101; numeric max:999999.9999 at 999999.9999 and 1000000; a pl-missing key falling back to en; validate_test.go cases for the min/between message fix.
(5) Docs in the same change (CLAUDE.md): modules/lagoon/README.md (API reference for ValidateRequest, RequestRule, Rule, ParseRules, CustomRule, UploadedFile, the supported rule list, the implicit-stop semantics), modules/phrasebook/README.md (the new lagoon::validation.* namespace beside lagoon::validate.*), docs/database/casts-and-validation.md (a "Request validation" section in prose; no hand-written Go fences, follow the 11.1 docs rules, use src= only for compiled examples). Every identifier named must exist.
go vet ./... && go test ./modules/lagoon -run '^(TestValidateRequest.*|TestValidate.Message.)$' -count=1 -v && go test ./modules/phrasebook -count=1 && go test ./cmd/summer -run TestDocsTree -count=1 && go -C ../fonoteka.go test ./plugins/golem15/user/... -count=1
<fails_when>Any command exits non-zero; the verbose run prints "no tests to run", "--- FAIL" or "--- SKIP", or lacks "--- PASS: TestValidateRequest"; TestDocsTree reports an unknown identifier or broken link; any user plugin test fails (core plugin regression).</fails_when>
<acceptance_criteria>
- go doc ./modules/lagoon ValidateRequest, go doc ./modules/lagoon ParseRules, go doc ./modules/lagoon CustomRule and go doc ./modules/lagoon UploadedFileFromHeader exit 0.
- grep -c 'Pole :attribute jest wymagane.' modules/phrasebook/lang/pl/validation.yaml prints 1 and grep -c 'max:' modules/phrasebook/lang/pl/validation.yaml prints at least 1.
- The en catalog holds after_or_equal and the pl catalog does not (grep -c '^after_or_equal' modules/phrasebook/lang/pl/validation.yaml prints 0), matching Winter's pl file.
- grep -c 'ValidateRequest' modules/lagoon/README.md and grep -c 'lagoon::validation' modules/phrasebook/README.md each print at least 1.
- A test asserts the exact pl string Pole posts jest wymagane. as the only message for {"posts":[]}.
- A validate_test.go case asserts that min:0 with -1 on an integer field answers with the min message carrying 0, not the max message.
</acceptance_criteria>
Any handler can hand decoded request input and PHP-ordered rules to one validator and get Laravel's exact errors object in the request locale; the old model validator keeps its contract with the min/between message fixed.
(2) webp, per D-24: go get golang.org/x/image@v0.46.0 in summercms.go (becomes a direct requirement), blank-import golang.org/x/image/webp in thumb.go beside the gif/jpeg/png decoders, and add a test decoding a small webp fixture with image.DecodeConfig and producing a crop thumb whose bytes are JPEG under the .webp name (document this in the attach section of modules/lagoon/README.md and docs/database/attachments.md). Then in ../fonoteka.go run go work sync and go mod tidy in the root module and both plugin modules so all go.sum and go.work.sum files list v0.46.0; go -C ../fonoteka.go build ./... must pass.
(3) tide multipart, per D-11: add Parts []Part to Request (yaml parts,omitempty), Part{Name, Value, File, Filename, ContentType, SHA256 string} where File is a path relative to the fixture directory (fixtures store upload bytes as files, e.g. files/cover.png), and a fixed exported boundary constant MultipartBoundary. multipart.go encodes parts in declaration order with mime/multipart using that boundary and sets Content-Type: multipart/form-data; boundary=... (overriding any recorded Content-Type header value only when it is a multipart type). Loading a flow checks every part file exists under BaseDir and its sha256 matches (mismatch is an error naming the part); a request with both Body and Parts is an error. RecordFlow and ReplayFlow send the same encoded bytes; variable substitution applies to Value and Path, never to file bytes. The recorder never inlines file bytes into YAML.
(4) Upload URL masking: in normalize.go, for leaf keys url and thumb_url whose string value starts with an uploads prefix, assert the shape <prefix>/<3 hex>/<3 hex>/<3 hex>/<disk> for originals and .../thumb_<digits>_<w>_<h>_0_0_<mode>.<ext> for thumbs (derive the partition and disk-name pattern from attach.PartitionDirectory and Winter getDiskName as read from vendor), then mask the partition and disk stem (keeping prefix, size, mode and extension visible); a wrong prefix, a missing partition or a different thumb size reports a Diff at the path, like the Carbon date check. Values that are not upload-shaped stay untouched. The prefix to accept is configurable on the normalizer (default /storage/app/uploads/public); never hard-code an application name.
(5) Publications: extend NormalizePublications so Carbon +00:00 values under *_at keys anywhere inside data.payload.album are masked with the same shape assertion as response bodies (A5 in RESEARCH: confirm first whether they are already masked; if they are, add only the test).
(6) Tests: multipart_test.go (round trip against an httptest server capturing the raw body: two recordings of the same flow produce identical bytes; a tampered part file fails load; Body plus Parts fails), normalize tests for upload URLs and publication dates. Docs in the same change: modules/tide/README.md (parts, boundary, upload masks, publication date masks), docs/services/parity-testing.md, docs/services/storage.md (the Winter layout recipe: bucket rooted at uploads/public with prefix /storage/app/uploads/public, and URL/PublicURL), docs/database/attachments.md (URL, webp).
go vet ./... && go test ./modules/lagoon/attach ./modules/tide -count=1 -v -run '^(TestFileURLWinterLayout|TestThumbWebP|TestMultipart.|TestNormalizeUploadURL.|TestNormalizePublication.*)$' && go test ./cmd/summer -run TestDocsTree -count=1 && go -C ../fonoteka.go build ./... && go -C ../fonoteka.go vet ./...
<fails_when>Any command exits non-zero; the verbose run prints "no tests to run", "--- FAIL" or "--- SKIP", or lacks a "--- PASS" line for TestFileURLWinterLayout, TestThumbWebP and a TestMultipart test; the fonoteka.go build fails on a go.sum mismatch for golang.org/x/image.</fails_when>
<acceptance_criteria>
- grep -c 'golang.org/x/image v0.46.0' go.mod prints 1 and the line has no // indirect comment.
- grep -c 'golang.org/x/image/webp' modules/lagoon/attach/thumb.go prints 1.
- go doc ./modules/lagoon/attach PublicURL and go doc ./modules/lagoon/attach File.URL exit 0.
- go doc ./modules/tide Part and go doc ./modules/tide MultipartBoundary exit 0.
- grep -c 'Parts' modules/tide/README.md prints at least 1 and grep -c 'storage/app/uploads/public' docs/services/storage.md prints at least 1.
- A tide test asserts that a thumb_url with a 100x100 size against a 200x200 expectation is reported as a Diff (masking never hides a wrong size).
</acceptance_criteria>
A multipart upload can be recorded from PHP and replayed against Go byte for byte, upload URLs are compared by shape without their random parts, the attach package emits Winter's URLs under the Winter layout config, and webp images decode.
(2) Docs in the same change: modules/beachcomber/README.md and docs/services/search.md (PageSearcher, SearchResult, the helper, QueryByWeights, the 250 per-page limit, and that ids remain candidates only that callers re-gate in SQL).
(3) User groups, per D-25 (sm-user-plugin, additive): models/user_group.go UserGroup{ID uint; Name string; Code *string; Description *string; Permissions *string; CreatedAt, UpdatedAt *time.Time} with TableName user_groups, registered in the models registry like the other models; add Groups []UserGroup to models.User with tag gorm:"many2many:users_groups;joinForeignKey:user_id;joinReferences:user_group_id" and json:"-" (GORM never writes it unless a caller associates groups; nothing in the user plugin does). Migration file updates/202610020001_create_user_groups.go (ID 202610020001_create_user_groups): create user_groups (id serial primary key, name varchar(255) not null, code varchar(255) null with an index, description text null, permissions text null, created_at and updated_at timestamp null) and users_groups (user_id integer not null, user_group_id integer not null, primary key (user_id, user_group_id) named user_group), then insert the Guest/guest and Registered/registered rows with PHP's descriptions; Rollback drops both tables. classes/user_groups.go: UserGroupCodes(ctx, db, userID uint) ([]string, error) (codes of the user's groups ordered by user_groups.id, null codes skipped) and HasGroupCode(ctx, db, userID uint, code string) (bool, error). The user API payload keeps "groups":[] exactly as today (D-25: contract unchanged); do not read the new table in any user-plugin handler.
(4) parity/schema_diff_test.go: add allowedDiffs entries user_groups and users_groups with a reason naming D-25 and that the frozen PHP snapshot predates the user-plugin dump (the same justification as user_throttle). Keep every other entry.
(5) Tests: updates/user_groups_test.go on the plugin's Postgres harness: migrate up creates both tables and the two seed rows, RollbackLast drops them, re-up works; UserGroupCodes returns admin for a user linked to a group with code admin and an empty slice for a user with no groups.
(6) Commits: commit the user-plugin files inside the submodule (git -C ../fonoteka.go/plugins/golem15/user), push its master, then commit parity/schema_diff_test.go together with the bumped submodule pointer in fonoteka.go, no co-author tags.
go vet ./... && go test ./modules/beachcomber/... -count=1 -v -run '^(TestSearchPage.|TestTypesenseSearchPage.)$' && go test ./cmd/summer -run TestDocsTree -count=1 && go -C ../fonoteka.go vet ./... && go -C ../fonoteka.go test ./plugins/golem15/user/updates -count=1 -v -run '^(TestUserGroups.*)$' && go -C ../fonoteka.go test ./parity -run '^(TestSchemaMatchesPHPSnapshot|TestUserAPINuxtFlows)$' -count=1
<fails_when>Any command exits non-zero; a verbose run prints "no tests to run", "--- FAIL" or "--- SKIP", or lacks "--- PASS" for a TestSearchPage test and a TestUserGroups test; TestSchemaMatchesPHPSnapshot reports an extra Go table; TestUserAPINuxtFlows fails (user payload changed).</fails_when>
<acceptance_criteria>
- go doc ./modules/beachcomber PageSearcher, go doc ./modules/beachcomber SearchPage and go doc ./modules/beachcomber Query.QueryByWeights exit 0.
- grep -c 'query_by_weights' modules/beachcomber/typesense/engine.go prints at least 1.
- grep -c 'PageSearcher' modules/beachcomber/README.md and grep -c 'PageSearcher' docs/services/search.md each print at least 1.
- grep -c '"user_groups"' ../fonoteka.go/parity/schema_diff_test.go and grep -c '"users_groups"' ../fonoteka.go/parity/schema_diff_test.go each print 1.
- grep -c 'json:"-"' ../fonoteka.go/plugins/golem15/user/models/user.go increases by one relative to HEAD (the Groups field is never serialized).
- grep '"groups":' ../fonoteka.go/plugins/golem15/user/controllers/api_controller.go | grep -c '\[\]any{}' prints 1, and this task's sm-user-plugin commit touches no file under controllers/ (payload untouched).
</acceptance_criteria>
Callers can ask the search engine for one page of candidate ids and the total it found, ranked with explicit weights, and any plugin can ask which group codes a user has without the user API changing.
<threat_model>
Trust Boundaries
| Boundary | Description |
|---|---|
| HTTP request body → request validator | Untrusted JSON and multipart input reaches rule evaluation, regex matching and DB exists lookups |
| Uploaded bytes → image decoders | Untrusted image bytes reach the gif/jpeg/png/webp decoders and the thumbnailer |
| Parity fixtures (git) → tide replay | Committed fixtures and part files drive requests; secrets must stay in the 0600 vars file |
| Search engine → application | Engine ids and counts are candidates, not authorization |
| Module proxy → go.mod | A dependency bump enters the build |
STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|---|---|---|---|---|---|
| T-12-14 | Tampering | lagoon exists: and regex rules |
high | mitigate | Table and column names pass identName; regex patterns come only from code (ParseRules at registration), never from input; an RE2-incompatible pattern fails at boot (Task 1). |
| T-12-15 | Denial of Service | wildcard expansion and size rules | medium | mitigate | Expansion is bounded by the decoded body, which surf caps by http.body_limits; size counts are O(n) over already-decoded values; no backtracking regex engine (RE2) (Task 1). |
| T-12-16 | Denial of Service | webp/png/jpeg decode in Thumb and DecodeConfig | medium | mitigate | DecodeConfig reads headers only; Thumb runs only on files that passed the 10240 KB cap in the callers (12-02/12-04); x/image v0.46.0 is the current upstream with its fixes (Task 2). |
| T-12-17 | Information Disclosure | tide multipart part files | medium | mitigate | Part bytes live as committed fixture files that are test images only; check_corpus --check-secrets still scans every YAML; substitution never touches file bytes (Task 2). |
| T-12-28 | Information Disclosure | beachcomber SearchPage found count | high | mitigate | Found is exposed to callers only as an input to a SQL recount (D-19, enforced in 12-04 and tested in 12-05); README states ids and counts are candidates (Task 3). |
| T-12-18 | Elevation of Privilege | user groups table | medium | mitigate | No route writes users_groups; Groups is never serialized; the site-admin predicate reads group codes server-side only (Task 3, consumed in 12-02). |
| T-12-SC | Tampering | Go module installs (golang.org/x/image v0.46.0) | high | mitigate | Official Go sub-repository already in the module graph, verified with go list -m against proxy.golang.org (RESEARCH Package Legitimacy Audit: OK); go.sum pins the hash; named by D-24 as the phase decision authorizing the bump. No npm/pip/cargo installs. |
| </threat_model> |
<success_criteria>
lagoon.ValidateRequestand the pl/enlagoon::validation.*catalogs exist and reproduce the recorded Polish 422 shape; lagoon.Validate callers are unchanged except the fixed min/between message.- attach exports URL helpers, decodes webp; tide records and replays multipart bodies and masks upload URLs and publication dates.
- beachcomber exposes found and weights; the user plugin has user groups with an unchanged user API.
- ROADMAP and REQUIREMENTS carry the D-03/D-04/D-06/D-19/D-20 wording. </success_criteria>