diff --git a/.planning/STATE.md b/.planning/STATE.md index 8e14df7..d9952be 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -5,10 +5,10 @@ current_phase: 11 current_phase_name: Jobs, realtime and search infrastructure status: planning stopped_at: Phase 16.3 complete, ready to plan Phase 11 -last_updated: "2026-10-11T12:08:30.918Z" +last_updated: "2026-10-11T12:28:12.226Z" last_activity: 2026-10-11 -last_activity_desc: Phase 16.3 complete, transitioned to Phase 11 -state_head: 5913337cdafd046f0d0d6d017b951144325e6b38 +last_activity_desc: "Completed quick task 261011-joh: sieve generic collection helpers module" +state_head: a0bf8ece050cc404041d1526d6db01d32aa8e1c2 progress: total_phases: 40 completed_phases: 13 @@ -33,7 +33,7 @@ See: .planning/PROJECT.md (updated 2026-09-16) Phase: 11 — Jobs, realtime and search infrastructure Plan: Not started Status: Ready to plan -Last activity: 2026-10-11 — Phase 16.3 complete, transitioned to Phase 11 +Last activity: 2026-10-11 - Completed quick task 261011-joh: sieve generic collection helpers module Progress: [████░░░░░░] 41% @@ -759,6 +759,7 @@ Recent decisions affecting current work: | 261009-mys | Add subtitle and avatar to backend users and a My account page (avatar menu, like WinterCMS) | 2026-10-09 | 143eca5, 64d21c8, 4f1cda3, 89611ea | [261009-mys-add-subtitle-and-avatar-to-backend-users](./quick/261009-mys-add-subtitle-and-avatar-to-backend-users/) | | 261010-iwu | Blog Editor live preview pane: toggleable, sandboxed iframe, embed thumbnail cards (16.2 follow-up) | 2026-10-10 | sm-journal-plugin df67939, c1f6a83, 2c78211 | [261010-iwu-blog-editor-live-preview-pane-toggleable](./quick/261010-iwu-blog-editor-live-preview-pane-toggleable/) | | 28 | Admin navbar View site link (admin origin root, new tab) | 2026-10-10 | 85fe541 | — | +| 261011-joh | sieve: generic collection helpers module (Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find, Set) | 2026-10-11 | 6933004, a0bf8ec | [261011-joh-sieve-generic-collection-helpers-module](./quick/261011-joh-sieve-generic-collection-helpers-module/) | ## Deferred Items diff --git a/.planning/quick/261011-joh-sieve-generic-collection-helpers-module/261011-joh-PLAN.md b/.planning/quick/261011-joh-sieve-generic-collection-helpers-module/261011-joh-PLAN.md new file mode 100644 index 0000000..f68f821 --- /dev/null +++ b/.planning/quick/261011-joh-sieve-generic-collection-helpers-module/261011-joh-PLAN.md @@ -0,0 +1,309 @@ +--- +phase: quick-261011-joh +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - modules/sieve/sieve.go + - modules/sieve/set.go + - modules/sieve/sieve_test.go + - modules/sieve/example_test.go + - modules/sieve/README.md + - README.md + - docs/architecture/introduction.md + - docs/setup/coming-from-wintercms.md +autonomous: true +requirements: [QUICK-261011-joh] + +estimate: + tokens: 70000 + raw_tokens: 70000 + tasks: 3 + confidence: low + +must_haves: + truths: + - "go doc -short ./modules/sieve lists exactly these exported names: Filter, Find, GroupBy, KeyBy, Map, Partition, Reduce, Set, SetOf, TryMap, Uniq, UniqBy. Set has the methods Has, Add and Slice. There is no Pluck (it would be Map under another name) and no Chunk (go1.27.0 already has slices.Chunk). None of the names duplicates a function in the standard library slices or maps packages (D-01, D-03)." + - "Every function is eager and returns a new slice or map. It never modifies its input, and its result never shares a backing array with the input, so changing or appending to a result leaves the caller's slice unchanged. Callbacks run once per element, in input order; TryMap and Find stop early (D-02)." + - "For nil or empty input, every slice- or map-returning function returns a non-nil empty value, so encoding/json writes [] or {} and never null. Find returns (zero, false), Reduce returns init, and SetOf() returns an empty set that Add can use (D-04)." + - "TryMap stops at the first error. It returns (nil, err) with the error unchanged (err == sentinel) and never calls f for the elements after the failing one (D-03)." + - "KeyBy keeps the later item when two items share a key. GroupBy keeps input order within each group and returns map[K]S, which keeps the caller's named slice type. Uniq and UniqBy keep the first occurrence in input order. Partition returns (matched, rest), both in input order. Find returns the first match (D-03)." + - "The non-test files of modules/sieve import nothing (go list shows no imports), the package has 100.0% statement coverage, and every exported function and every Set method has a runnable Example with an // Output: line." + - "modules/sieve/README.md follows the module README structure with the exact summary and **Name:** lines. The root README modules table has the sieve row between postcard and sunrise. docs/architecture/introduction.md and docs/setup/coming-from-wintercms.md mention sieve. TestDocsTree, docs:build --check, go vet ./... and go test -short ./... pass. The work lands as two code commits staged by explicit path, with no co-author trailer and no change to any other module (D-05 to D-08)." + artifacts: + - path: modules/sieve/sieve.go + provides: "Package doc stating the contracts; Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find" + contains: "func TryMap[" + - path: modules/sieve/set.go + provides: "type Set[T comparable] map[T]struct{}, SetOf, Set.Has, Set.Add, Set.Slice" + contains: "func SetOf[" + - path: modules/sieve/sieve_test.go + provides: "Plain testing-package unit tests at 100.0% statement coverage: nil input, ordering, non-aliasing, input immutability, TryMap early stop, KeyBy later-wins, named slice types, Set behaviour" + contains: "func TestTryMap" + - path: modules/sieve/example_test.go + provides: "A runnable Example for every exported function and Set method; ExampleMap shows the pluck use" + contains: "func ExampleMap" + - path: modules/sieve/README.md + provides: "Module README: H1, summary, import line, Name line, then Overview, Role in the framework, Features, Usage, API reference, Dependencies, Testing" + contains: "**Name:** A sand sieve" + - path: README.md + provides: "Modules table row for sieve, between postcard and sunrise" + contains: "[sieve](modules/sieve/README.md)" + - path: docs/architecture/introduction.md + provides: "Utilities row in the framework modules table" + contains: "[sieve](../../modules/sieve/README.md)" + - path: docs/setup/coming-from-wintercms.md + provides: "A concept-map row mapping Laravel collection methods to sieve, and the rewritten not-provided row for the fluent collection object" + contains: "sieve.KeyBy" + key_links: + - from: modules/sieve/README.md + to: internal/docsite (TestDocsTree, docs:build --check) + via: "loadModules turns every modules/ with a non-test Go file into an API page. splitReadme needs '# sieve' followed by a summary line, and checkSpan resolves every sieve.X code span against the package." + pattern: "^# sieve$" + - from: README.md + to: modules/sieve/README.md + via: "The modules table row reuses the summary sentence as the purpose and the Name text as the mnemonic" + pattern: "\\[sieve\\]\\(modules/sieve/README.md\\)" + - from: docs/setup/coming-from-wintercms.md + to: modules/sieve/README.md + via: "The concept-map row maps Laravel collection methods to sieve identifiers; the link and identifier checkers resolve them" + pattern: "sieve.GroupBy" + - from: modules/sieve/example_test.go + to: modules/sieve/sieve.go + via: "Example functions with // Output: lines run under go test and render in go doc" + pattern: "// Output:" +--- + + +Add a new framework module, `modules/sieve`: generic, zero-dependency collection helpers for slices and maps, covering what the Go 1.27 standard library `slices` and `maps` packages do not provide. That means Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition and Find, plus a small `Set` type built with `SetOf`. The module ships with a README in the standard module structure, a row in the root README modules table, mentions on the two fitting docs pages, Example tests and full unit tests. + +Purpose: plugin and framework code that ports Laravel collection pipelines (`map`, `filter`, `keyBy`, `groupBy`, `unique`, `partition`, `first`, `pluck`) gets one typed, reviewed implementation instead of each plugin writing its own loops. All results are non-nil, so encoded API bodies keep `[]`/`{}` rather than `null`. + +Output: modules/sieve (sieve.go, set.go, sieve_test.go, example_test.go, README.md), one new root README row, and edits to two docs pages. It lands as two commits: the tracer, then the rest of the API with its docs. No existing module changes. + + + +@~/.claude/msd-core/workflows/execute-plan.md +@~/.claude/msd-core/templates/summary.md + + + +@CLAUDE.md +@README.md +@modules/towel/README.md +@modules/towel/example_test.go +@docs/architecture/introduction.md +@docs/setup/coming-from-wintercms.md + +Locked decisions (from the user, cited as D-NN in the tasks): +- D-01: The toolchain is Go 1.27 (local go1.27.0). Do not duplicate the standard library: every planned name was checked against `go doc slices` and `go doc maps`. `slices.Chunk` exists (`func Chunk[Slice ~[]E, E any](s Slice, n int) iter.Seq[Slice]`), so sieve ships no Chunk, and the README points to `slices.Chunk` instead. The module has zero dependencies: its non-test files import nothing. +- D-02: The API is eager and slice-based. Functions return new slices or maps and never mutate their input. iter.Seq variants are out of scope. +- D-03: Signatures are idiomatic and generic: Map[S ~[]T, T, U any]; TryMap returns ([]U, error) and stops at the first error; Find returns (T, bool); Partition returns (matched, rest); KeyBy and GroupBy take a key func returning a comparable K; Reduce(s, init, f) returns A; Uniq keeps first-occurrence order, with a UniqBy variant. Pluck and SetOf were left to the planner (see PD-1 and PD-4). +- D-04: Nil input gives an empty, non-nil result. Document this the same way everywhere. +- D-05: The README structure is H1 name, the one-sentence summary, the import line, the `**Name:**` mnemonic line, then the sections. The root README modules table gets a row, in alphabetical order, whose purpose text reuses the summary and whose third column is the mnemonic. +- D-06: Every identifier named in a README or docs page must exist in the package; the docs checker enforces this. Framework READMEs never name a consuming application. +- D-07: Commits carry no co-author trailer of any kind. The user's CLAUDE.md rule overrides any attribution reminder. Code and planning docs go in separate commits, and files are staged by explicit path, because other sessions may be working in this repo. +- D-08: One plan with 1 to 3 tasks. Example tests and full unit tests land in the same task as the code. No existing module is refactored. Add a docs mention where a fitting page exists. The gates are `go test ./cmd/summer -run TestDocsTree`, `summer docs:build --check`, `go vet ./...` and `go test -short ./...`. + +Planner decisions (Claude's discretion, recorded): +- PD-1: Pluck is dropped as redundant. Laravel's `pluck('title')` takes a field name as a string, which in Go would need reflection and would give up compile-time checking. The typed equivalent is Map with a field accessor. The README Usage section says this and shows the Map form, and the Map doc comment and ExampleMap show the pluck use. +- PD-2: Chunk is dropped because of D-01. The README names `slices.Chunk` (lazy) and `slices.Collect(slices.Chunk(s, n))` (eager). +- PD-3: UniqBy is added. Deduplicating structs by an ID is the common case, and Uniq only works for comparable element types. +- PD-4: `type Set[T comparable] map[T]struct{}` is a defined map type, so `len`, `range`, `delete` and `maps.Keys` work on it directly. It has three value-receiver methods: `Has(v T) bool`, `Add(items ...T)` and `Slice() []T`. Slice returns elements in unspecified order and is non-nil even for a nil Set. `SetOf[T comparable](items ...T) Set[T]` is variadic, so `SetOf("a", "b")` and `SetOf(ids...)` both work, including for named slice types. A nil Set answers Has and Slice. Add on a nil Set panics like any nil-map write; document this and point to SetOf. +- PD-5: Functions whose element type does not change keep the caller's named slice type. Filter, Uniq, UniqBy and Partition return S, and GroupBy returns map[K]S. Map and TryMap return []U. +- PD-6: TryMap returns (nil, err) on failure with err unchanged, not wrapped. A caller who wants the index wraps the error inside f. +- PD-7: In KeyBy, a later item overwrites an earlier one with the same key, as Laravel's keyBy does. +- PD-8: The README includes a "Role in the framework" section between Overview and Features. All 24 module READMEs have one since commit 602e041 ("docs: add a Role in the framework section to every module README"), so it is part of the standard structure. +- PD-9: Docs mentions. docs/architecture/introduction.md gets a new "Utilities" row in the framework modules table, since no existing concern fits. docs/setup/coming-from-wintercms.md gets a concept-map row for Laravel collections, and its "What is not provided" row for Collections is rewritten to cover only the fluent collection object, pointing to sieve. That row currently tells readers to use only `slices` and `maps`, which is out of date once sieve exists. +- PD-10: The work lands as two commits. The tracer commit (Task 1) is followed by one commit holding the rest of the API, its tests and README, and both docs edits (Tasks 2 and 3). The docs edits ride in the same change as the API they describe, per CLAUDE.md. + +Exact strings (copy verbatim; the gates match them byte for byte): +- SUMMARY (README line 3, and the purpose cell of the root row): Generic, dependency-free collection helpers that the standard library slices and maps packages lack: mapping, filtering, reducing, keying, grouping, deduplicating, partitioning and finding, plus a small set type. +- NAME (README line 7): **Name:** A sand sieve: you shake a collection through it and keep what you need; map, filter and reduce for slices and maps. +- ROOT_ROW (one line in README.md, directly after the postcard row and directly before the sunrise row): | [sieve](modules/sieve/README.md) | Generic, dependency-free collection helpers that the standard library slices and maps packages lack: mapping, filtering, reducing, keying, grouping, deduplicating, partitioning and finding, plus a small set type. | A sand sieve: you shake a collection through it and keep what you need; map, filter and reduce for slices and maps. | +- INTRO_ROW (docs/architecture/introduction.md, appended after the "Console and tooling" row): | Utilities | [sieve](../../modules/sieve/README.md) | + +Facts the planner verified (no need to re-derive): +- `go version` is go1.27.0. The `go doc slices` functions are All, AppendSeq, Backward, BinarySearch(Func), Chunk, Clip, Clone, Collect, Compact(Func), Compare(Func), Concat, Contains(Func), Delete(Func), Equal(Func), Grow, Index(Func), Insert, IsSorted(Func), Max(Func), Min(Func), Repeat, Replace, Reverse, Sort(Func), SortStableFunc, Sorted(Func), SortedStableFunc and Values. The `go doc maps` functions are All, Clone, Collect, Copy, DeleteFunc, Equal(Func), Insert, Keys and Values. None of them is Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find, Set or SetOf. Compact removes only consecutive duplicates, in place; DeleteFunc filters in place; IndexFunc returns an index, not the element. +- The signatures in were compiled and vetted in a scratch module under go1.27.0. Type inference works from the slice argument alone; `SetOf(ids...)` with `type IDs []int` compiles; `GroupBy` on an IDs value yields `map[bool]IDs`; ExampleSet_Has and ExampleSet_Slice, which name methods on a generic type, pass `go vet`'s tests analyzer; `fmt.Println` of a map prints keys sorted, so Example output for KeyBy and GroupBy is deterministic. +- `go doc -short ./` prints top-level funcs and types. A constructor such as SetOf is listed indented under its type, which the gate's `sed -E 's/^ *(func|type) ([A-Za-z]+).*/\2/'` handles. `go list -f '{{join .Imports " "}}'` prints an empty line for a package with no imports; test imports are not included. +- internal/docsite: every modules/ directory with a non-test .go file must have a README.md whose first line is `# ` followed by a summary line; otherwise TestDocsTree fails. Code spans of the forms `pkg.Ident`, `pkg.Type.Member`, `pkg.Ident(...)` and `pkg.Ident[...]` are checked when pkg is a module package name. `slices.X` and `maps.X` spans are ignored, because they are not modules. Write methods as `sieve.Set.Has`, not `sieve.Set[T].Has`. Go code fences in module READMEs need no src= (only docs/ pages do). docs/ page headings must be plain ASCII with no code or links. Any spelling of the consuming application's name is rejected. +- Baseline on master at b906bdd: `go run ./cmd/summer docs:build --check` prints "docs:build: no problems found" in about 4 s, and `go vet ./...` is green. The only untracked path is `.msd/`; never stage it. +- Baseline `go test -short ./...` takes about 55 s. Inside the planning sandbox, cabana, conga, lagoon and lighthouse failed only with a Docker socket "permission denied" error on unix:///var/run/docker.sock (testcontainers tests that do not skip under -short). Those are environment failures, not regressions; see Task 3 for how to handle them. +- Module tests in this repo use the plain `testing` package (no testify): internal `package ` tests plus a `package _test` example_test.go, as in modules/towel. +- Commit prefixes are module-scoped, for example `feat(cabana): ...` and `fix(admin): ...`. + + + +Exact exported API of modules/sieve (compiled in a scratch module under go1.27.0; parameter names are recommendations): + +```go +// sieve.go +func Map[S ~[]T, T, U any](s S, f func(T) U) []U +func TryMap[S ~[]T, T, U any](s S, f func(T) (U, error)) ([]U, error) +func Filter[S ~[]T, T any](s S, keep func(T) bool) S +func Reduce[S ~[]T, T, A any](s S, init A, f func(A, T) A) A +func KeyBy[S ~[]T, T any, K comparable](s S, key func(T) K) map[K]T +func GroupBy[S ~[]T, T any, K comparable](s S, key func(T) K) map[K]S +func Uniq[S ~[]T, T comparable](s S) S +func UniqBy[S ~[]T, T any, K comparable](s S, key func(T) K) S +func Partition[S ~[]T, T any](s S, match func(T) bool) (matched, rest S) +func Find[S ~[]T, T any](s S, match func(T) bool) (T, bool) + +// set.go +type Set[T comparable] map[T]struct{} +func SetOf[T comparable](items ...T) Set[T] +func (s Set[T]) Has(v T) bool +func (s Set[T]) Add(items ...T) +func (s Set[T]) Slice() []T +``` + + + + + + Task 1: Tracer: the sieve module with Map, wired through its README, the root modules table and the docs checker + modules/sieve/sieve.go, modules/sieve/example_test.go, modules/sieve/README.md, README.md + modules/towel/README.md, modules/towel/example_test.go, README.md (the Modules table, lines 90 to 117), internal/docsite/load.go (loadModules and splitReadme) + +Prove the one path every new module takes through the framework: a Go package, a runnable example, the module README, docs-site ingestion with the identifier check, and the root modules table. Only Map is built here; Task 2 adds the rest. + +1. Create modules/sieve/sieve.go in package sieve, with a package doc comment. Its first sentence is "Package sieve provides generic, dependency-free collection helpers that the standard library slices and maps packages lack." A second paragraph states the contracts per D-02 and D-04. Every function is eager and never modifies its input. Results never share a backing array with the input. Callbacks run once per element in input order. Nil or empty input yields a non-nil empty slice or map, so encoding/json writes [] or {} rather than null. Add only Map, with the signature in interfaces. Allocate the result with make([]U, 0, len(s)), so it is non-nil even for a nil s, and call f once per element in order. Its doc comment says that Map is also the pluck operation, extracting a field through an accessor func (PD-1). The file has no import block at all (D-01). + +2. Create modules/sieve/example_test.go in package sieve_test, importing fmt and git.golem15.com/golem15/summercms/modules/sieve. Add ExampleMap: a local `post` struct with ID int and Title string, three values, then sieve.Map extracting the titles (the pluck use), printed with fmt.Println and checked by an // Output: line. Use only neutral names such as blog and acme (D-06). + +3. Create modules/sieve/README.md in the towel layout (D-05, PD-8): + - Line 1 is "# sieve", then a blank line. + - Line 3 is the SUMMARY string from context, verbatim, then a blank line. + - Line 5 is the import line in a code span, formatted like towel's (import "git.golem15.com/golem15/summercms/modules/sieve"), then a blank line. + - Line 7 is the NAME string from context, verbatim. + - Then these sections, in this order: Overview, Role in the framework, Features, Usage, API reference, Dependencies, Testing. + At this stage: + - The API reference table (columns Identifier and Description) has exactly one row, which starts with "| `sieve.Map` |". + - Features and Usage cover Map only. Usage has a short Go block in package blog that maps posts to titles. + - Dependencies lists three bullets: SummerCMS modules: none; Third-party: none; Standard library: none (the package has no imports). + - Testing shows go test ./modules/sieve/... in an sh block and says the tests are in-memory and need no external services. + Write no sieve.X code span for any identifier that Task 2 will add. TestDocsTree resolves every such span against the package as it stands at this commit. + +4. Edit README.md at the repo root with Edit, never Write. Insert ROOT_ROW from context as one line directly after the postcard row and directly before the sunrise row of the Modules table. Change nothing else. + +5. Run every check in verify up to and including `git diff --quiet -- go.mod go.sum`; all must pass. Then stage exactly the four paths by name (git add modules/sieve/sieve.go modules/sieve/example_test.go modules/sieve/README.md README.md). Never use git add -A or git add ., and never stage .msd/. Commit with the subject "feat(sieve): add sieve module with Map" and no trailer (D-07). Finally, run the whole verify command, including the commit checks. + + + cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./modules/sieve && go test -count=1 -run 'ExampleMap' -v ./modules/sieve | grep -q -- '--- PASS: ExampleMap' && test -z "$(go list -f '{{join .Imports " "}}' ./modules/sieve)" && go test ./cmd/summer -run 'TestDocsTree|TestDocsBuildRealTree' -count=1 && go run ./cmd/summer docs:build --check && test "$(sed -n 1p modules/sieve/README.md)" = '# sieve' && test "$(sed -n 3p modules/sieve/README.md)" = 'Generic, dependency-free collection helpers that the standard library slices and maps packages lack: mapping, filtering, reducing, keying, grouping, deduplicating, partitioning and finding, plus a small set type.' && grep -qxF '**Name:** A sand sieve: you shake a collection through it and keep what you need; map, filter and reduce for slices and maps.' modules/sieve/README.md && grep -qxF '| [sieve](modules/sieve/README.md) | Generic, dependency-free collection helpers that the standard library slices and maps packages lack: mapping, filtering, reducing, keying, grouping, deduplicating, partitioning and finding, plus a small set type. | A sand sieve: you shake a collection through it and keep what you need; map, filter and reduce for slices and maps. |' README.md && awk '/^\| \[postcard\]/{p=NR} /^\| \[sieve\]/{s=NR} /^\| \[sunrise\]/{u=NR} END{exit !(p && s==p+1 && u==s+1)}' README.md && git diff --quiet -- go.mod go.sum && TRACER=$(git log --format=%H -1 --grep='^feat(sieve): add sieve module with Map$') && test -n "$TRACER" && MSG=$(git log -1 --format=%B "$TRACER") && ! printf '%s' "$MSG" | grep -qi 'co-authored' && FILES=$(git show --name-only --format= "$TRACER") && test "$(printf '%s\n' "$FILES" | LC_ALL=C sort | tr '\n' ' ')" = 'README.md modules/sieve/README.md modules/sieve/example_test.go modules/sieve/sieve.go ' + + modules/sieve exists with Map and no imports. ExampleMap passes. The README has the exact summary and Name lines and all seven sections in order. The root README row sits between postcard and sunrise. TestDocsTree, TestDocsBuildRealTree and docs:build --check pass. The commit with the subject "feat(sieve): add sieve module with Map", found by its subject rather than by a HEAD offset, touches exactly those four files and has no trailer. + + + + Task 2: The rest of the API with full unit tests, Example tests and the complete README + modules/sieve/sieve.go, modules/sieve/set.go, modules/sieve/sieve_test.go, modules/sieve/example_test.go, modules/sieve/README.md + modules/sieve/sieve.go, modules/sieve/example_test.go, modules/sieve/README.md (all from Task 1), modules/towel/context_test.go (test style) + + - Map: nil input gives a non-nil, length-0 result. Order is preserved, f is called exactly len(s) times, and changing the result leaves the input unchanged. + - TryMap: on success, every element is mapped in order, and nil input gives ([]U{} non-nil, nil). When f fails at index i (cases i = 0 and i = 2 of 4), the result is (nil, sentinel) with err == sentinel and errors.Is true, and f was called exactly i+1 times. + - Filter: keeps matches in input order. Nil input, or no matches, gives a non-nil empty result. When every element matches, the result has a new backing array (changing result[0] leaves the input unchanged). The input equals a slices.Clone snapshot afterwards. A named type `type ids []int` comes back as ids (checked at compile time by assigning to an ids variable). + - Reduce: sums ints. Nil input returns init unchanged. String concatenation proves left-to-right order ("abc"). The accumulator type may differ from the element type (int elements into a string or map accumulator). + - KeyBy: on duplicate keys the later item wins. Nil input gives a non-nil empty map. key is called once per element. + - GroupBy: each group keeps input order, the result type is map[K]S for a named S, and nil input gives a non-nil empty map. Appending to one group changes neither the input nor any other group. + - Uniq: [3 1 3 2 1] gives [3 1 2]. Input with no duplicates still gets a new backing array. Nil input gives a non-nil empty result, and the input is unchanged. + - UniqBy: deduplicating structs by an ID field keeps the first occurrence, in order. key is called once per element. + - Partition: matched and rest both keep input order. Nil input gives two non-nil empty slices. When every element matches, rest is non-nil and empty. + - Find: returns the first match and true. Without a match, or for nil input, it returns (zero value, false). The predicate is not called after the first match. + - Set: SetOf deduplicates, and SetOf() is non-nil and accepts Add. Has reports presence and absence. A nil Set has Has false and a non-nil, length-0 Slice. Add is variadic. Slice holds each element exactly once (sort, then compare). len and delete work directly on a Set. + - `go test -cover ./modules/sieve` reports exactly 100.0% of statements. + + +Implement the rest of interfaces per D-02, D-03 and D-04 and PD-3 to PD-7, tests first. + +1. RED. Write modules/sieve/sieve_test.go in package sieve (internal tests, plain testing package, no testify, table-driven where it reads naturally), covering every behavior bullet. A compile failure for the missing identifiers counts as the failing run. Confirm it fails with go test ./modules/sieve. + +2. GREEN, in modules/sieve/sieve.go. Add TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition and Find with the exact signatures in interfaces. + - Every slice-returning path allocates its own result with make (never s[:0], never an alias of s), so the result is non-nil and independent of the input (D-02, D-04). Map-returning paths use make(map...). + - TryMap returns nil and the unwrapped error from f at the first failure (PD-6). + - KeyBy lets later items win (PD-7). + - Uniq and UniqBy track seen values or keys in a map[T]struct{} or map[K]struct{} and keep the first occurrence. + - Find returns the zero value and false when nothing matches. + - Each function gets a Go doc comment that states its nil-input result and, where it applies, ordering, early stop and duplicate-key behaviour. + - No imports in any non-test file (D-01). + +3. GREEN, in a new file modules/sieve/set.go: Set, SetOf, Has, Add and Slice per PD-4. All methods use value receivers. SetOf sizes its map from len(items). Slice returns make([]T, 0, len(s)) filled by ranging the map. The type's doc comment says that Set is a plain map, that Slice order is unspecified (use slices.Sorted(maps.Keys(set)) for ordered output when T is ordered), and that Add on a nil Set panics, so callers should create sets with SetOf. + +4. Extend modules/sieve/example_test.go with one Example each: ExampleTryMap (strconv.Atoi over strings, once with all valid and once with a bad value, printing the error), ExampleFilter, ExampleReduce, ExampleKeyBy, ExampleGroupBy, ExampleUniq, ExampleUniqBy, ExamplePartition, ExampleFind, ExampleSetOf, ExampleSet_Has, ExampleSet_Add and ExampleSet_Slice. ExampleSet_Slice sorts with slices.Sort before printing. Print maps with fmt.Println, which sorts the keys. Each Example has an // Output: line. Test-file imports such as strconv and slices are fine, because only non-test files must have none. + +5. Complete modules/sieve/README.md, keeping lines 1 to 7 and the seven-section order from Task 1: + - Overview: what sieve is and its WinterCMS/Laravel counterpart (collection methods on Eloquent results), plus the four contracts from the package doc. + - Role in the framework: a dependency-free leaf with no imports, so any module, plugin or application can use it without import cycles. It complements slices and maps rather than wrapping them. Then a "Use cases:" list: shaping API responses from models, keying records by ID to join two result sets, grouping, deduplicating IDs before an IN query, and set membership checks. + - Features: one bullet per function or group, naming each identifier as a sieve.X code span. + - Usage: one Go block in package blog with a Post struct (ID, Author, Title, Published) that uses Filter, Map, KeyBy, GroupBy, UniqBy, Partition, Find, TryMap and SetOf. After it, a paragraph explaining why there is no Pluck (PD-1): Laravel's pluck takes a field name as a string, which would need reflection and lose type checking, so the typed equivalent is the inline span `sieve.Map(posts, func(p Post) string { return p.Title })`. Then a short list headed "What the standard library already covers" pointing to slices.Chunk (with slices.Collect(slices.Chunk(s, n)) for an eager [][]T), slices.Contains, slices.IndexFunc, slices.Sort and slices.SortFunc, slices.Min and slices.Max, slices.Compact (consecutive duplicates only, in place), slices.DeleteFunc (in-place filter), and maps.Keys and maps.Values (with slices.Sorted for ordered output), each in a code span (PD-2, D-01). + - API reference: one row per identifier, each row starting with "| `sieve.` |", for exactly these 15: Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find, Set, SetOf, Set.Has, Set.Add, Set.Slice. Each description gives the nil-input result and any ordering or duplicate rule. + - Dependencies: still none, none, none. + - Testing: the sh block plus a sentence that the tests are in-memory, include runnable examples and cover every statement. + Every sieve.X span must resolve (D-06). Do not commit in this task: the change is committed together with Task 3's docs edits (PD-10). + + + cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./modules/sieve && go test -count=1 -cover ./modules/sieve | grep -q 'coverage: 100.0% of statements' && test -z "$(go list -f '{{join .Imports " "}}' ./modules/sieve)" && test "$(go doc -short ./modules/sieve | sed -E 's/^ *(func|type) ([A-Za-z]+).*/\2/' | LC_ALL=C sort | tr '\n' ' ')" = 'Filter Find GroupBy KeyBy Map Partition Reduce Set SetOf TryMap Uniq UniqBy ' && go doc -c ./modules/sieve Set.Has >/dev/null && go doc -c ./modules/sieve Set.Add >/dev/null && go doc -c ./modules/sieve Set.Slice >/dev/null && test "$(go test -count=1 -list 'Example.*' ./modules/sieve | grep -c '^Example')" -ge 14 && test "$(grep '^## ' modules/sieve/README.md | tr '\n' '|')" = '## Overview|## Role in the framework|## Features|## Usage|## API reference|## Dependencies|## Testing|' && test "$(for id in Map TryMap Filter Reduce KeyBy GroupBy Uniq UniqBy Partition Find Set SetOf Set.Has Set.Add Set.Slice; do grep -qF "| \`sieve.$id\` |" modules/sieve/README.md || echo "$id"; done)" = '' && grep -qF '`slices.Chunk`' modules/sieve/README.md && go test ./cmd/summer -run 'TestDocsTree|TestDocsBuildRealTree' -count=1 && go run ./cmd/summer docs:build --check && git diff --quiet -- go.mod go.sum + + All 12 exported names and the 3 Set methods exist with the signatures in interfaces. Coverage is 100.0%, there are no imports, and every behavior bullet has a passing test. At least 14 Example functions with output pass. The README has all 15 API reference rows, the Pluck explanation and the standard-library list, and TestDocsTree and docs:build --check pass. Nothing is committed yet. + + + + Task 3: Docs mentions, the full gates and the feature commit + docs/architecture/introduction.md, docs/setup/coming-from-wintercms.md + docs/architecture/introduction.md (the framework modules table, lines 21 to 32), docs/setup/coming-from-wintercms.md (the concept map table and the "What is not provided" table) + +1. In docs/architecture/introduction.md, use Edit to append INTRO_ROW from context as a new last row of the "The framework modules" table, directly after the "Console and tooling" row (PD-9). Change nothing else. + +2. In docs/setup/coming-from-wintercms.md (PD-9, D-06), use Edit: + a. Concept map table: insert one row directly after the "Eloquent models" row. The WinterCMS cell names Laravel collections and the methods `map`, `filter`, `reduce`, `keyBy`, `groupBy`, `unique`, `partition`, `first` and `pluck` as bare code spans. The SummerCMS cell says these work on plain slices and maps with `sieve.Map`, `sieve.TryMap`, `sieve.Filter`, `sieve.Reduce`, `sieve.KeyBy`, `sieve.GroupBy`, `sieve.Uniq`, `sieve.Partition` and `sieve.Find`. It adds that pluck is `sieve.Map` with a field accessor, unique-by-key is `sieve.UniqBy`, `chunk` is the standard `slices.Chunk`, and sets are `sieve.SetOf`. The Where cell is [sieve](../../modules/sieve/README.md). There is no guide page, so the module reference stands alone. + b. "What is not provided" table: rewrite the row whose first cell is Collections. Its first cell becomes the fluent collection object (the Collection class and method chaining). Its second cell says this is not provided, and that you call the [sieve](../../modules/sieve/README.md) functions and the standard `slices` and `maps` packages on plain slices and maps. Keep the table's existing sentence style. + Do not add Go code fences: docs/ pages require src= on Go fences. Do not add headings. + +3. Run the verify checks up to and including `go test -short ./...`. If go test -short ./... fails only in cabana, conga, lagoon or lighthouse with a Docker socket "permission denied" error on unix:///var/run/docker.sock (the sandbox blocks the Docker socket; this was observed at planning time and is not a regression), rerun the same command outside the sandbox. If Docker is still unreachable, run the suite without those four packages, record in the SUMMARY which packages were skipped and why, and continue. Any other failure is a real failure: fix it before committing. + +4. Commit Task 2 and Task 3 together (PD-10, D-07). Stage exactly these seven paths by name: docs/architecture/introduction.md, docs/setup/coming-from-wintercms.md, modules/sieve/README.md, modules/sieve/example_test.go, modules/sieve/set.go, modules/sieve/sieve.go and modules/sieve/sieve_test.go. Never use git add -A or git add ., and never stage .msd/. Use the subject "feat(sieve): add the remaining collection helpers and docs mentions". Add a body listing TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find, Set and SetOf, plus the two docs pages, and add no trailer. Then run the whole verify command, including the commit checks and the check that no other module changed (D-08). + + + cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && grep -qxF '| Utilities | [sieve](../../modules/sieve/README.md) |' docs/architecture/introduction.md && grep -qF '[sieve](../../modules/sieve/README.md)' docs/setup/coming-from-wintercms.md && grep -qF '`sieve.KeyBy`' docs/setup/coming-from-wintercms.md && grep -qF '`sieve.GroupBy`' docs/setup/coming-from-wintercms.md && grep -qF '`sieve.UniqBy`' docs/setup/coming-from-wintercms.md && grep -qF '`slices.Chunk`' docs/setup/coming-from-wintercms.md && ! grep -q '^| Collections |' docs/setup/coming-from-wintercms.md && go vet ./... && go test ./cmd/summer -run 'TestDocsTree|TestDocsBuildRealTree' -count=1 && go run ./cmd/summer docs:build --check && go test -count=1 -cover ./modules/sieve | grep -q 'coverage: 100.0% of statements' && go test -short ./... && git diff --quiet -- go.mod go.sum && TRACER=$(git log --format=%H -1 --grep='^feat(sieve): add sieve module with Map$') && FEAT=$(git log --format=%H -1 --grep='^feat(sieve): add the remaining collection helpers and docs mentions$') && test -n "$TRACER" && test -n "$FEAT" && git merge-base --is-ancestor "$TRACER" "$FEAT" && MSG=$(git log -1 --format=%B "$FEAT") && ! printf '%s' "$MSG" | grep -qi 'co-authored' && FILES=$(git show --name-only --format= "$FEAT") && test "$(printf '%s\n' "$FILES" | LC_ALL=C sort | tr '\n' ' ')" = 'docs/architecture/introduction.md docs/setup/coming-from-wintercms.md modules/sieve/README.md modules/sieve/example_test.go modules/sieve/set.go modules/sieve/sieve.go modules/sieve/sieve_test.go ' && TFILES=$(git show --name-only --format= "$TRACER") && test "$(printf '%s\n' "$TFILES" | LC_ALL=C sort | tr '\n' ' ')" = 'README.md modules/sieve/README.md modules/sieve/example_test.go modules/sieve/sieve.go ' + + The architecture intro has the Utilities row. The WinterCMS page maps Laravel collection methods to sieve identifiers, and its not-provided row now covers only the fluent collection object. go vet ./..., TestDocsTree, TestDocsBuildRealTree, docs:build --check, 100.0% sieve coverage and go test -short ./... all pass, with any Docker-sandbox skip recorded in the SUMMARY. There are two feat(sieve) commits, found by subject; the tracer is an ancestor of the feature commit. Neither has a trailer, and each touches exactly its listed files, so no module outside modules/sieve changed. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| request data → sieve helpers | Plugin handlers may pass request-derived slices (IDs from a query string, decoded JSON arrays) straight through Map, TryMap, Uniq, KeyBy or SetOf | +| sieve results → API responses | Results are encoded by encoding/json into bodies that must stay byte-compatible with the PHP backend | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-joh-01 | Tampering (aliasing: a result sharing the caller's backing array, so a later append or write corrupts cached or shared data) | sieve.go Filter, Uniq, UniqBy, Partition, GroupBy | medium | mitigate | Every result is allocated with make and never re-slices s. Task 2 tests change results (including the all-match and no-duplicate cases) and assert the input still equals a slices.Clone snapshot, and they check that appending to one GroupBy group leaves the input and the other groups unchanged. | +| T-joh-02 | Tampering (parity drift: null instead of [] or {} in encoded bodies) | all slice- and map-returning functions, Set.Slice | low | mitigate | Nil or empty input yields non-nil empty results (D-04). Task 2 tests assert non-nil for every function on nil input. The README and doc comments state the rule. | +| T-joh-03 | Information Disclosure / Tampering (nondeterministic order leaking into responses) | Set.Slice, KeyBy, GroupBy map results | low | mitigate | Set.Slice is documented as unordered and points to slices.Sorted(maps.Keys(set)). Map results are encoded by encoding/json with sorted keys. Order-preserving functions (Filter, Uniq, UniqBy, Partition, GroupBy groups) have ordering tests. | +| T-joh-04 | Denial of Service (large request-derived input) | all functions | low | accept | Output size is at most linear in input size (Map and TryMap 1:1; the others at most n), and allocation is a single pass. Callers bound input size through surf body limits; sieve adds no amplification. | +| T-joh-05 | Repudiation / error masking (TryMap swallowing or rewriting errors) | sieve.go TryMap | low | mitigate | TryMap returns f's error unchanged and stops at the first one. Task 2 asserts err == sentinel and the call count. | +| T-joh-SC | Tampering | npm/pip/cargo installs | low | accept | No package is added. Each task verifies an empty import list for modules/sieve and `git diff --quiet -- go.mod go.sum`. | + + + +- go vet ./... is green and go test -short ./... passes (Docker-sandbox handling as in Task 3). +- go test ./cmd/summer -run 'TestDocsTree|TestDocsBuildRealTree' -count=1 and go run ./cmd/summer docs:build --check report no problems. +- modules/sieve has 100.0% statement coverage, no non-test imports and exactly the exported names in interfaces, with at least 14 passing Examples. +- The root README row sits between postcard and sunrise, and the README has the exact summary and Name lines. +- Two feat(sieve) commits with explicitly staged files and no trailer; no change outside modules/sieve except README.md and the two docs pages. + + + +A developer can import git.golem15.com/golem15/summercms/modules/sieve and use Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find and Set/SetOf on plain slices with full type inference and no new dependency. The module's README, the root modules table and two docs pages describe it, the docs checker validates every identifier, and nothing in the standard library is duplicated. + + + +Create `.planning/quick/261011-joh-sieve-generic-collection-helpers-module/261011-joh-SUMMARY.md` when done. + diff --git a/.planning/quick/261011-joh-sieve-generic-collection-helpers-module/261011-joh-SUMMARY.md b/.planning/quick/261011-joh-sieve-generic-collection-helpers-module/261011-joh-SUMMARY.md new file mode 100644 index 0000000..6993fdf --- /dev/null +++ b/.planning/quick/261011-joh-sieve-generic-collection-helpers-module/261011-joh-SUMMARY.md @@ -0,0 +1,105 @@ +--- +phase: quick-261011-joh +plan: 01 +subsystem: modules/sieve +status: complete +tags: [go, generics, collections, module, docs] +requires: [] +provides: + - "modules/sieve: Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find, Set, SetOf (Set.Has, Set.Add, Set.Slice)" +affects: + - README.md (modules table) + - docs/architecture/introduction.md + - docs/setup/coming-from-wintercms.md +tech-stack: + added: [] + patterns: + - "Eager generic helpers over S ~[]T that allocate their own result with make, so results are non-nil and never alias the input" + - "Element-preserving functions return the caller's named slice type S (GroupBy returns map[K]S)" +key-files: + created: + - modules/sieve/sieve.go + - modules/sieve/set.go + - modules/sieve/sieve_test.go + - modules/sieve/example_test.go + - modules/sieve/README.md + modified: + - README.md + - docs/architecture/introduction.md + - docs/setup/coming-from-wintercms.md +decisions: + - "sieve ships no Pluck (Map with a field accessor) and no Chunk (slices.Chunk exists in Go 1.27)" + - "TryMap returns (nil, err) with f's error unwrapped; KeyBy lets the later item win" + - "Set is a defined map type with value-receiver Has/Add/Slice; Add on a nil Set panics, so SetOf is the constructor" +metrics: + duration: 5m + completed: 2026-10-11 + tasks: 3 + files: 8 +actuals: + tokens: 8300 + tasks: 3 + commits: 2 +plan_head_before: b906bdd48bb361608e6c926113c322e62e5c9238 +plan_head_after: a0bf8ece050cc404041d1526d6db01d32aa8e1c2 +--- + +# Phase quick-261011-joh Plan 01: sieve generic collection helpers module Summary + +Zero-import generic collection helpers (Map, TryMap, Filter, Reduce, KeyBy, GroupBy, Uniq, UniqBy, Partition, Find and a map-backed Set built with SetOf). Every function is eager, never aliases its input, and returns non-nil empty results for nil input, so JSON bodies keep `[]`/`{}`. The package has 100.0% statement coverage and 14 runnable Examples, and is documented in its README, the root modules table and two docs pages. + +## Tasks + +| Task | Name | Commit | Files | +|------|------|--------|-------| +| 1 | Tracer: sieve module with Map, README, root modules table row | 6933004 | modules/sieve/sieve.go, modules/sieve/example_test.go, modules/sieve/README.md, README.md | +| 2 | Rest of the API with unit tests, Examples and the complete README (TDD) | a0bf8ec | modules/sieve/sieve.go, modules/sieve/set.go, modules/sieve/sieve_test.go, modules/sieve/example_test.go, modules/sieve/README.md | +| 3 | Docs mentions and full gates | a0bf8ec | docs/architecture/introduction.md, docs/setup/coming-from-wintercms.md | + +Tasks 2 and 3 share one commit, as the plan specifies (PD-10). + +## Verification + +- Tracer gate: after commit 6933004 the Task 1 verify ran again end to end (go vet, ExampleMap, empty import list, TestDocsTree and TestDocsBuildRealTree, docs:build --check, the exact README lines, root row placement between postcard and sunrise, commit file set, no trailer). It passed. +- TDD: RED was a compile failure of sieve_test.go (undefined: TryMap, Filter, ...). GREEN came after sieve.go and set.go were added. No refactor step was needed. +- `go test -cover ./modules/sieve`: coverage 100.0% of statements. `go list` shows no imports for the package. `go doc -short` lists exactly Filter Find GroupBy KeyBy Map Partition Reduce Set SetOf TryMap Uniq UniqBy, and `go doc` resolves Set.Has, Set.Add and Set.Slice. There are 14 Example functions, and all pass. +- `go vet ./...`: green. +- `go test ./cmd/summer -run 'TestDocsTree|TestDocsBuildRealTree'` and `go run ./cmd/summer docs:build --check`: passed ("no problems found"). +- `git diff --quiet -- go.mod go.sum`: unchanged. +- I also confirmed by hand that the README Usage Go block compiles and vets, using a throwaway directory that was then removed. +- Commit checks: both feat(sieve) commits were found by subject, the tracer is an ancestor of the feature commit, neither has a co-author trailer, and each touches exactly its listed files. + +### go test -short ./... and Docker (NOT passing for 5 packages) + +`go test -short ./...` fails in **beachcomber, cabana, conga, lagoon and lighthouse**, and only because testcontainers cannot reach the Docker daemon: + +``` +docker info: permission denied while trying to connect to the docker API at unix:///var/run/docker.sock +``` + +- The failure is the same outside the Claude sandbox. `/var/run/docker.sock` is `root:docker` mode 0660, and user `golem` (uid 1002) is not in the `docker` group, so Docker is not reachable from this host session at all. +- beachcomber, conga, lagoon and lighthouse fail in TestMain while starting the shared Postgres container. In cabana, 48 FAIL lines all trace to the same Docker error: 44 tests fail directly and 4 are parent tests of Docker-failed subtests. No assertion line in any of these packages is anything other than the Docker error. +- The plan named four packages (cabana, conga, lagoon, lighthouse). beachcomber shows the same Docker-only failure. +- None of these five packages imports sieve, and this change does not touch them. +- The remaining 36 packages pass under `go test -short` with these five excluded (exit 0). +- **These five packages were not verified in this run** and should be re-run on a host with Docker access. + +## Deviations from Plan + +None. The plan was executed as written. The only difference is the Docker handling above: beachcomber also needed excluding, and Docker was unreachable outside the sandbox too. + +Note: the commits landed directly on `master`, as the orchestrator instructed (main tree, no worktree isolation). Nothing was pushed or tagged. + +## Known Stubs + +None. + +## Threat Flags + +None. No network, auth, file or schema surface was added. The plan's threat register mitigations are covered: tests for non-aliasing and input immutability (T-joh-01), tests that nil input gives non-nil results (T-joh-02), ordering tests and the documented unordered Set.Slice (T-joh-03), and a TryMap test that the sentinel comes back unchanged and f stops being called (T-joh-05). + +## Self-Check: PASSED + +- FOUND: modules/sieve/sieve.go, set.go, sieve_test.go, example_test.go, README.md +- FOUND: commit 6933004 (feat(sieve): add sieve module with Map) +- FOUND: commit a0bf8ec (feat(sieve): add the remaining collection helpers and docs mentions)