Files
summercms/.planning/phases/12.1-user-plugin-admin-screens/12.1-VALIDATION.md
Jakub Zych 93f0171e9c docs(12.1-05): security review and validation sign-off for Phase 12.1
- 12.1-SECURITY-REVIEW.md: every threat T-12.1-01 to T-12.1-40 and T-12.1-SC with its mitigation, test and observed result; T-12-18 revisited; the D-30 guard and its boundary; the eleven handed-over items; five findings that need a decision
- 12.1-VALIDATION.md: per-task map with real task ids and measured run times, signed off
- deferred-items.md: older framework files that name an application
2026-10-05 16:13:50 +02:00

16 KiB

phase, slug, status, nyquist_compliant, wave_0_complete, created, validated, gate, removal_gate
phase slug status nyquist_compliant wave_0_complete created validated gate removal_gate
12.1 user-plugin-admin-screens validated true true 2026-10-04 2026-10-05 scripts/check-phase12.1.sh --all scripts/check-phase12.1.sh --removal

Phase 12.1 — Validation Strategy

Per-phase validation contract for feedback sampling during execution. Seeded from 12.1-RESEARCH.md § Validation Architecture and finalized by plan 12.1-05. The phase has no requirement IDs; success criteria (SC) and CONTEXT.md decisions (D-NN) are the units.


Test Infrastructure

Property Value
Framework Go testing + testcontainers Postgres (cabana and plugin harnesses); SPA: vitest + @vue/test-utils + happy-dom
Config file admin/vitest.config.ts; Go needs none
Quick run command (framework) go vet ./modules/cabana/ ./modules/pact/ && go test ./modules/cabana/... ./modules/pact/... -count=1
Quick run command (plugin) go -C ../fonoteka.go test ./plugins/golem15/user/... -count=1
Quick run command (SPA) npm --prefix admin run typecheck && npm --prefix admin test
Full suite command (framework) go vet ./... && go test ./... -count=1
Full suite command (application) scripts/check-phase12.1.sh --app (vet and tests of every module of the application workspace; go -C ../fonoteka.go test ./... alone covers the root module only)
Docs go test ./cmd/summer -run TestDocsTree -count=1 and go run ./cmd/summer docs:build --check
Generated artefacts scripts/check-admin-openapi.sh --check and scripts/check-admin-dist.sh
Phase gate scripts/check-phase12.1.sh --all, and scripts/check-phase12.1.sh --removal on its own

Measured run times (2026-10-05, one workstation, warm build cache)

Command Time
go test ./modules/cabana -run '^TestPhase121Threats$' -count=1 11 s
go test ./modules/cabana -count=1 63 s
go -C ../fonoteka.go test ./plugins/golem15/user/... -count=1 40 s (package user 37 s, classes 39 s, updates 11 s, in parallel)
npm --prefix admin run typecheck && npm --prefix admin test 52 s (71 files, 1013 tests)
scripts/check-phase12.1.sh --self-test 2 s
scripts/check-phase12.1.sh --go 87 to 130 s
scripts/check-phase12.1.sh --security 84 s
scripts/check-phase12.1.sh --coverage 111 s
scripts/check-phase12.1.sh --spa 52 s
scripts/check-phase12.1.sh --openapi 11 s
scripts/check-phase12.1.sh --dist 19 to 22 s
scripts/check-phase12.1.sh --docs 8 s
scripts/check-phase12.1.sh --hygiene under 1 s
scripts/check-phase12.1.sh --app 290 to 325 s (five workspace modules)
scripts/check-phase12.1.sh --evidence under 1 s
scripts/check-phase12.1.sh --all 11 min 47 s
scripts/check-phase12.1.sh --removal 4 min 13 s for 26 rows (27 rows now)

Sampling Rate

  • After every task commit: the quick run command of the repository the task wrote to; plus scripts/check-admin-openapi.sh --check and scripts/check-admin-dist.sh when admin/ or swag annotations changed
  • After every plan wave: both full suites and the docs checks
  • Before /gsd-verify-work: scripts/check-phase12.1.sh --all green, and --removal green on the same heads
  • Feedback latency: a quick run answers in under 65 s in each repository (measured above); the stated maximum for a task-level check is 2 minutes, which every quick run and every single gate stage except --app meets. The whole gate takes about 12 minutes and is a per-wave check, not a per-task one.

Per-Task Verification Map

Task ids are <plan>-T<task>. Framework commands run in summercms.go; plugin commands run inside the application workspace. Every command below is also run by a stage of scripts/check-phase12.1.sh.

Task ID Plan Wave Requirement Threat Ref Secure Behavior Test Type Automated Command File Exists Status
12.1-01-T1 01 1 D-09 / SC-1 T-12.1-01, T-12.1-02, T-12.1-03, T-12.1-05 Bulk action ids outside the list scope never reach Run; a partial selection is refused; an undeclared or unregistered action is 404; the action's own permission is enforced; one transaction, rollback on error; a YAML bulk action the controller does not register fails boot integration + unit + SPA go test ./modules/cabana -run '^(TestBulkAction|TestListSchemaBulkActions|TestPhase121BootErrors)' -count=1; npm --prefix admin test -- tests/list/BulkActionsMenu tests/list/ListToolbar tests/list/ListView ✅ ✅ green
12.1-01-T2 01 1 D-10 / SC-1 T-12.1-04 A record action runs in the form scope: out of scope 404, not applicable 409, own permission enforced, only offered actions in meta.actions integration + SPA go test ./modules/cabana -run '^TestRecordAction' -count=1; npm --prefix admin test -- tests/form/RecordActions ✅ ✅ green
12.1-01-T3 01 1 D-12 / SC-1 T-12.1-07 Row state from the fixed set; an unknown value is dropped by the server and renders nothing in the SPA integration + SPA go test ./modules/cabana -run '^TestRowState' -count=1; npm --prefix admin test -- tests/list/RowStateBadges tests/list/DataTable ✅ ✅ green
12.1-01-T4 01 1 D-27 (G4) / SC-3 T-12.1-06, T-12.1-08 A hook or action refuses with a readable 403 and a rollback; any other error is the opaque 500; one log line per action run integration + SPA go test ./modules/cabana -run '^(TestForbidden|TestSoftDeletedRecord|TestPhase121Threats)' -count=1; npm --prefix admin test -- tests/form/FormErrorBanner tests/form/FormView ✅ ✅ green
12.1-02-T1 02 2 D-11 / SC-1 T-12.1-14, T-12.1-15, T-12.1-16 A context: preview field is never writable; the status hint is served as allowlisted nodes; a preview URL from YAML stays on the current controller integration + SPA go test ./modules/cabana -run '^TestPreview' -count=1; npm --prefix admin test -- tests/form/PreviewView tests/form/PreviewField tests/app/winterUrl tests/app/router ✅ ✅ green
12.1-02-T2 02 2 D-19, D-27 (G1, G2, G5, G7), D-28 / SC-3 T-12.1-09, T-12.1-10 Password and virtual fields are never bound, filled or returned; rules per operation replace the model rules; preset shapes integration + SPA go test ./modules/cabana -run '^(TestPasswordField|TestVirtualFields|TestFormRules|TestPreset)' -count=1; npm --prefix admin test -- tests/form/PasswordField tests/form/formState ✅ ✅ green
12.1-02-T3 02 2 D-16 / SC-2 T-12.1-13 permissioneditor: unknown code 422, value outside the mode's set 422, changed locked code 403, stored codes that are not offered kept integration + SPA go test ./modules/cabana -run '^TestPermissionEditor' -count=1; npm --prefix admin test -- tests/form/PermissionEditorField ✅ ✅ green
12.1-02-T4 02 2 D-07, D-27 (G3, G6) / SC-3 T-12.1-11, T-12.1-12 A protected foreign key is writable only with the opt-in; locked relation ids cannot be added or removed on create or update; invisible columns are searched and not sent integration + SPA go test ./modules/cabana -run '^(TestRelationLock|TestWritableForeignKey|TestInvisibleColumn|TestFilterOptions)' -count=1; npm --prefix admin test -- tests/form/RelationField tests/list/DataTable ✅ ✅ green
12.1-02-T5 02 2 D-25 T-12.1-17 The owner decides how the contract is released before a tag exists decision checkpoint answered tag-local on 2026-10-05 (12.1-02-SUMMARY.md) n/a ✅ green
12.1-02-T6 02 2 contract / SC-4 T-12.1-17 New routes in the inventory, the permission matrix and the OpenAPI document; the tag is on a head where the gates passed contract go test ./modules/cabana -run '^(TestPhase09ContractInventory|TestPhase09PermissionMatrix|TestPhase10OpenAPIConformance)$' -count=1; scripts/check-admin-openapi.sh --check; scripts/check-admin-dist.sh ✅ ✅ green
12.1-03-T1 03 3 SC-1, D-30 T-12.1-18, T-12.1-38, T-12.1-39 The Users screen needs its permission; without the extra permission, changing the email or password of a privileged-group member, or permanently deleting them (form delete, bulk delete as a whole), is 403 and nothing changes; with it each succeeds; a name-only update stays allowed integration go -C ../fonoteka.go test ./plugins/golem15/user -run '^(TestAdminUsersTracer|TestAdminPrivilegedMember|TestPhase121Threats)$' -count=1 ✅ ✅ green
12.1-03-T2 03 3 SC-3 / D-13 / D-14 T-12.1-23, T-12.1-30 activate, unban, unsuspend, deactivate, restore, ban; permanent delete with cleanup; no action writes users_groups integration go -C ../fonoteka.go test ./plugins/golem15/user -run '^(TestAdminUserActions|TestAdminUserForceDelete)$' -count=1; go -C ../fonoteka.go test ./plugins/golem15/user/classes -run '^TestAdminActions$' -count=1 ✅ ✅ green
12.1-03-T3 03 3 D-15, D-19, D-20 T-12.1-19, T-12.1-20, T-12.1-21, T-12.1-22, T-12.1-27 Password mismatch 422; a reset ends sessions; send_invite sends one mail; no secret in a response; the avatar is shared with the user API; the resolver equals PHP getMergedPermissions on a table of cases integration + unit go -C ../fonoteka.go test ./plugins/golem15/user -run '^(TestAdminUserPassword|TestAdminUserInvite|TestAdminAvatarSharedWithAPI|TestAdminUserFormFields)$' -count=1; go -C ../fonoteka.go test ./plugins/golem15/user/classes -run '^(TestMergedPermissions|TestPermissionSetScan)$' -count=1 ✅ ✅ green
12.1-03-T4 03 3 D-17, migrations T-12.1-24, T-12.1-25 last_seen is written by the auth path at most once per five minutes, never fails it, and is absent from every user payload; three additive migrations up, down and up integration go -C ../fonoteka.go test ./plugins/golem15/user -run '^TestLastSeen$' -count=1; go -C ../fonoteka.go test ./plugins/golem15/user/updates -count=1 ✅ ✅ green
12.1-04-T1 04 4 SC-2, SC-4, D-04, D-07 T-12-18, T-12.1-28, T-12.1-30 The groups field writes exactly the chosen memberships; without the extra permission a privileged membership change is 403 and nothing changes, on create too; every other path leaves users_groups unchanged integration go -C ../fonoteka.go test ./plugins/golem15/user -run '^(TestAdminPrivilegedGroups|TestAdminUserGroupsField)$' -count=1 ✅ ✅ green
12.1-04-T2 04 4 SC-1, D-06, D-24, D-29 T-12.1-29, T-12.1-31 User Groups screen; a privileged code cannot be created, re-coded or deleted without the extra permission; the list counts members integration go -C ../fonoteka.go test ./plugins/golem15/user -run '^(TestAdminGroups|TestAdminGroupsEdge|TestAdminGroupsControllerGuards)$' -count=1 ✅ ✅ green
12.1-04-T3 04 4 SC-1, SC-2, D-22 T-12.1-32 Organisations screen; the members manager sets and clears organisation_id and nothing else integration go -C ../fonoteka.go test ./plugins/golem15/user -run '^(TestAdminOrganisations|TestAdminOrganisationMembers|TestAdminOrganisationsEdge)$' -count=1 ✅ ✅ green
12.1-04-T4 04 4 SC-4, D-08, schema T-12.1-33, T-12.1-34 The Go schema matches the PHP snapshot with one allow-list entry; a user with groups marshals and answers the user API without them; the recorded plugin pointer is the plugin head integration + regression go -C ../fonoteka.go test ./parity -run '^(TestSchemaMatchesPHPSnapshot|TestUserAPINuxtFlows|TestParityCorpus)$' -count=1; the application's TestPhase12Threats (subtest T-12-18); scripts/check-phase12.1.sh --app ✅ ✅ green
12.1-05-T1 05 5 SC-5, D-04 to D-08, D-30 T-12.1-35, T-12.1-37 One named threat test per repository with a subtest per mitigated threat; the gate refuses a failure, a skip, zero tests and a missing named test integration + gate scripts/check-phase12.1.sh --self-test && scripts/check-phase12.1.sh --security ✅ ✅ green
12.1-05-T2 05 5 SC-5 — Every Go behaviour of the phase has a unit or integration test; coverage of pact, cabana and the plugin's five packages is at least 80 percent each unit + integration scripts/check-phase12.1.sh --go && scripts/check-phase12.1.sh --coverage ✅ ✅ green
12.1-05-T3 05 5 SC-5, docs T-12.1-36, T-12.1-40, T-12.1-SC The SPA behaviours and the five UI backstops are unit-tested; docs identifiers, links and snippets hold; no application name in framework files; no dependency change; every high or critical protection is load-bearing SPA + checker + gate scripts/check-phase12.1.sh --all; scripts/check-phase12.1.sh --removal ✅ ✅ green

Status: ✅ green · ❌ red · ⚠️ flaky

Measured coverage (statements, scripts/check-phase12.1.sh --coverage, 2026-10-05): modules/pact 100.0%, modules/cabana 86.6%; the plugin's packages root 89.1%, classes 89.0%, controllers 83.0%, models 95.7%, updates 90.2%. The floor is 80 percent per package.


Wave 0 Requirements

  • A neutral cabana fixture plugin (modules/cabana/testdata/roster, acme.roster) with bulk actions, record actions, a preview field, row state and a permission editor (plans 01 and 02)
  • sm-user-plugin admin test harness: admin_harness_test.go boots the plugin with cabana mounted and mints a backend principal with chosen permissions (plan 03)
  • SPA fixtures under admin/tests/fixtures/ for the new schema fields: roster.list-schema.json, roster.list.json, roster.record.json, roster.form-schema.json (plans 01 and 02)
  • scripts/check-phase12.1.sh (plan 05)

Framework install: none needed. No Go module and no npm package was added or changed in the phase.


Manual-Only Verifications

One end-of-phase human check, from plan 12.1-05 Task 3 (workflow.human_verify_mode is end-of-phase). It is not automated because visual fit with the design system in both themes cannot be asserted by unit tests.

  • Test: start the application against the tagged framework, sign in as a backend admin holding golem15.users.access_users and golem15.users.access_groups but not golem15.users.manage_privileged_groups, and walk the three screens in light and dark mode: filter and search Users, open a banned and a deactivated user's preview, run Activate, Unban and a bulk Ban, create a user with an invitation, open the Permissions tab, try to add the admin group to a user, edit a group's permissions, add and remove an organisation member. Then open a user who is in the admin group: change the name only and save; change the email and save; enter a new password and save; press Delete; select that user together with another one and use the bulk delete.
  • Expected: the screens match the UI-SPEC (row-state badges with text, one status callout on the preview, record actions before the single primary edit button, the segmented permission control, the locked admin group with its note, the forbidden banner when the locked group is forced through a crafted request), and nothing in the app's own user payloads changed. For the user in the admin group (D-30): the name-only save succeeds; the email save and the password save each show the forbidden banner with the marked field, keep what was typed and save nothing; Delete and the bulk delete each show a danger toast and delete nobody.

Validation Sign-Off

  • All tasks have <automated> verify or Wave 0 dependencies (12.1-02-T5 is a decision checkpoint and has none by design)
  • Sampling continuity: no 3 consecutive tasks without automated verify
  • Wave 0 covers all MISSING references
  • No watch-mode flags
  • Feedback latency recorded and within the stated maximum
  • nyquist_compliant: true set in frontmatter

Approval: validated 2026-10-05 by plan 12.1-05 (gsd-executor), on scripts/check-phase12.1.sh --all and --removal passing; the manual check above is left to /gsd-verify-work.