docs(14.3-04): complete unit tests and phase gate plan

2026-10-08 12:31:45 +02:00
parent c4c91f0f6b
commit 9dac7b8f67
1 changed files with 235 additions and 0 deletions
--- a/.planning/phases/14.3-one-command-app-scaffolding-summer-new-and-summercms-io-new/14.3-04-SUMMARY.md
+++ b/.planning/phases/14.3-one-command-app-scaffolding-summer-new-and-summercms-io-new/14.3-04-SUMMARY.md
@@ -0,0 +1,235 @@
+---
+phase: 14.3-one-command-app-scaffolding-summer-new-and-summercms-io-new
+plan: 04
+subsystem: testing
+tags: [golden-tests, summer-new, bootstrap-script, phase-gate, security-review, validation]
+
+requires:
+  - phase: 14.3-01
+    provides: "build.NewApp, planNewApp, frameworkFromBinary seam, --generator gate stage"
+  - phase: 14.3-02
+    provides: "modules/sunrise new.sh and Script(), --acceptance stage, refusal probe, 14.3-SECURITY-REVIEW.md"
+  - phase: 14.3-03
+    provides: "site TestTerminalCommands with SUMMERCMS_NEW_URL/REPO/REF overrides, docs"
+provides:
+  - "internal/build/new_golden_test.go: -update flag, 17 goldens for the default app and 6 for --no-plugin"
+  - "18 generator/CLI/key tests (TestNew*, TestMakePluginRoutesUnchanged, TestGenerateAppKey, TestNewCommandViaCLI)"
+  - "TestScriptBehavior (25 cases on a stub PATH against a bare fake framework remote) and TestScriptShellcheck"
+  - "scripts/check-phase14.3.sh --unit, --forbidden, --bootstrap, --terminal, --security, --all"
+  - "14.3-SECURITY-REVIEW.md with T-14.3-01..26 + SC pinned; 14.3-VALIDATION.md validated"
+affects: [v0.1.13 release, site require bump and redeploy, /gsd-verify-work 14.3]
+
+actuals:
+  tokens: 30074
+  tasks: 3
+  commits: 6
+plan_head_before: 9f56dc7eff3a569dae61496fb8997f9f8d18bde8
+plan_head_after: c4c91f0f6b41610be8db25bfcbf1eccc1cf788b8
+
+tech-stack:
+  added: []
+  patterns:
+    - "Golden files named from the app-relative path ('/' -> '__', leading '.' -> 'dot-') so .env goldens stay clear of the framework's .env ignore rules"
+    - "Script behaviour harness: real git through a symlink, bash stubs for go/docker/summer recording calls, cwd, args and stdin; a bare remote fed from a work clone so tags can move"
+    - "Gate stages run as separate processes from --all so each stage's EXIT trap (Docker cleanup) fires before the next stage"
+    - "Tests that make git commits in temp dirs disable detached auto maintenance (GIT_CONFIG_COUNT maintenance.auto=false, gc.auto=0)"
+
+key-files:
+  created:
+    - internal/build/new_golden_test.go
+    - internal/build/testdata/new/ (17 goldens)
+    - internal/build/testdata/new-no-plugin/ (6 goldens)
+    - modules/sunrise/script_behavior_test.go
+  modified:
+    - internal/build/new_test.go
+    - internal/build/build_test.go
+    - modules/lagoon/keygen_test.go
+    - cmd/summer/main_test.go
+    - scripts/check-phase14.3.sh
+    - .planning/phases/14.3-one-command-app-scaffolding-summer-new-and-summercms-io-new/14.3-SECURITY-REVIEW.md
+    - .planning/phases/14.3-one-command-app-scaffolding-summer-new-and-summercms-io-new/14.3-VALIDATION.md
+
+key-decisions:
+  - "Golden app key is base64 of bytes 0x00..0x1f (goldenAppKey); the existing testAppKey stays for the other tests"
+  - "Goldens use framework auto-detection through a summercms sibling symlink, so the golden replace is ../summercms and the sibling path is exercised"
+  - "TestGenerateAppKeyIs32ByteBase64 renamed to TestGenerateAppKey (the name the gate requires); body unchanged"
+  - "The gate also requires named PASS lines for the site tests (TestNewScriptRoute, TestRoutesAssemble, TestLoadTerminal, TestTerminalScript, TestTerminalEnv, TestTerminalRunScript) and fails when a golden changed during the run"
+  - "--security parses the threat table (escaped pipes respected): every ID present, no open row, every high row mitigated and pinned by a TestX name or a gate stage"
+
+patterns-established:
+  - "Phase gate --all: one process per stage, named PASS per required test, final marker line printed once"
+
+requirements-completed: [D-01, D-02, D-03, D-04, D-05, D-06, D-07, D-08, D-09, D-10, D-11, D-12, D-13, D-14, D-15, D-16, D-17, D-18, D-19, D-20, D-21, D-22, D-23, D-24, D-25, D-26, D-27, D-28]
+
+coverage:
+  - id: D1
+    description: "Every file summer new writes is pinned byte for byte for the default and --no-plugin apps, with an explicit -update path"
+    requirement: D-21
+    verification:
+      - kind: unit
+        ref: "internal/build/new_golden_test.go#TestNewGolden"
+        status: pass
+      - kind: unit
+        ref: "internal/build/new_golden_test.go#TestNewGoldenNoPlugin"
+        status: pass
+    human_judgment: false
+  - id: D2
+    description: "Generator error and behaviour matrix: name rule and keywords, existing targets, framework resolution order, inside-checkout refusal, cleanup on failure, key validation, .env 0600, AGENTS.md symlink, git first commit and identity paths, --no-git, --no-plugin real build"
+    requirement: D-02
+    verification:
+      - kind: unit
+        ref: "internal/build/new_test.go#TestNewValidateAppName,TestNewRefusesExistingTarget,TestNewFrameworkResolution,TestNewRefusesTargetInsideFramework,TestNewCleansUpOnFailure,TestNewRequiresValidAppKey"
+        status: pass
+      - kind: unit
+        ref: "internal/build/new_test.go#TestNewEnvFileMode,TestNewAgentsSymlink,TestNewGitFirstCommit,TestNewGitUsesConfiguredIdentity,TestNewGitMissingIsWarning,TestNewNoGit"
+        status: pass
+      - kind: integration
+        ref: "internal/build/new_test.go#TestNewAppSmoke,TestNewNoPluginBuilds"
+        status: pass
+    human_judgment: false
+  - id: D3
+    description: "Shared-path and CLI pins: relative go.work from plugin:add, plain make:plugin routes unchanged, GenerateAppKey, summer new through the CLI root"
+    requirement: D-27
+    verification:
+      - kind: unit
+        ref: "internal/build/build_test.go#TestMakeAndAddPluginOnHelloCopy,TestMakePluginRoutesUnchanged"
+        status: pass
+      - kind: unit
+        ref: "modules/lagoon/keygen_test.go#TestGenerateAppKey"
+        status: pass
+      - kind: unit
+        ref: "cmd/summer/main_test.go#TestNewCommandViaCLI,TestToolCommandNames"
+        status: pass
+    human_judgment: false
+  - id: D4
+    description: "Bootstrap script behaviour matrix (25 cases) and shellcheck"
+    requirement: D-16
+    verification:
+      - kind: unit
+        ref: "modules/sunrise/script_behavior_test.go#TestScriptBehavior"
+        status: pass
+      - kind: unit
+        ref: "modules/sunrise/script_behavior_test.go#TestScriptShellcheck"
+        status: pass
+    human_judgment: false
+  - id: D5
+    description: "Fail-closed phase gate across summercms.go and the site: unit, generator, bootstrap, terminal, acceptance, security"
+    requirement: D-21
+    verification:
+      - kind: e2e
+        ref: "bash scripts/check-phase14.3.sh --all (Phase 14.3 gate passed, 2026-10-08)"
+        status: pass
+    human_judgment: false
+  - id: D6
+    description: "Security review T-14.3-01..26 + SC pinned and validation map signed off"
+    verification:
+      - kind: other
+        ref: "bash scripts/check-phase14.3.sh --security"
+        status: pass
+    human_judgment: true
+    rationale: "Self-performed review; a human should read the register, the accepted risks and the pins"
+  - id: D7
+    description: "Release steps (push, tag v0.1.13, site require bump, redeploy, production acceptance)"
+    requirement: D-28
+    verification: []
+    human_judgment: true
+    rationale: "User actions after the phase; listed below, nothing pushed, tagged or deployed"
+
+duration: 19min
+completed: 2026-10-08
+status: complete
+---
+
+# Phase 14.3 Plan 04: Unit tests, phase gate and sign-off Summary
+
+**Golden files pin every file `summer new` writes, 18 named tests cover each generator, CLI and key branch, 25 stub-PATH cases pin the bootstrap script's selection and refusal paths, and `bash scripts/check-phase14.3.sh --all` (vet, full suite with named PASS lines, generator on PostgreSQL 16, gated bootstrap and landing-page runs, container acceptance, threat register) ends with `Phase 14.3 gate passed`.**
+
+## Performance
+
+- **Duration:** 19 min
+- **Started:** 2026-10-08T10:11:15Z
+- **Completed:** 2026-10-08T10:30:44Z
+- **Tasks:** 3
+- **Files modified:** 32 (23 goldens, 6 test files, the gate, 2 planning docs)
+
+## Accomplishments
+
+- Golden harness (`-update`, fixed 0x00..0x1f key, `summercms` sibling symlink): go.mod, go.work, summer.yaml, main.go, plugins.gen.go, config/*, .env, .env.example, .gitignore, compose.yaml, CLAUDE.md, the starter plugin's plugin.go and routes.go, and next steps; six goldens for `--no-plugin`. Reviewed by eye against D-03, D-10, D-23 and D-25. A one-byte edit made `TestNewGolden` fail with a diff (checked by hand, restored).
+- Generator matrix in `internal/build/new_test.go`: all 25 Go keywords plus `main`/`internal`, four existing-target kinds left untouched, eight framework-resolution subtests (flag wins, wrong flag module never falls through, `summercms` before `summercms.go`, foreign sibling skipped, binary source last, nothing found names `--framework`), cleanup leaving siblings intact, key validation before any filesystem access, git identity paths, missing git, `--no-git`, `--no-plugin` real build.
+- `TestScriptBehavior`: help/usage/name errors (including a shell-metacharacter name), existing app, Go missing/1.26.3/devel, git missing, Docker missing, fresh clone at v0.10.0 (numeric order, v1.0.0-rc1 ignored) with `summer new demo --framework <abs>` from the parent and stdin not read, `SUMMERCMS_REF`, dash ref, forward move to v0.11.0, no-op re-run, dirty/branch/non-fast-forward/moved-tag/foreign-module/non-git refusals with HEAD unchanged, PATH hint on/off, GOPATH/bin fallback. A mutation run (forced fetch, stdin left open) made three cases fail.
+- `scripts/check-phase14.3.sh --all` passed: 28 required framework tests plus 6 site tests with named PASS lines, generator stage, `TestBootstrapEndToEnd`, `TestTerminalCommands`, node:24-bookworm acceptance (`{"message":"Hello from SummerCMS","plugin":"golem15.demo"}`) and the security check. No `phase143*` or `summercms-terminal-*` container or volume remained, and the site checkouts stayed clean.
+- Fail-closed checks were exercised on planted inputs: an open high row, a missing T-14.3-26 and an unpinned SC row each fail `--security`, and a skipped required subtest, a missing PASS line or "no tests to run" each fail `require_named_pass`.
+
+## Task Commits
+
+1. **Task 1: golden harness** - `eed24b8` (test)
+2. **Task 2: generator, CLI and key matrix** - `392dd3c` (test)
+3. **Task 3: script matrix, gate, sign-off** - `93e9a8a` (test), `89f1ab2` (chore, gate), `d0af79b` (docs, security review and validation), `c4c91f0` (chore, unique gate marker)
+
+The tracer gate after Task 1 (interactive, end-of-phase, automated-only verify) re-ran the verify chain, which passed, so expansion continued.
+
+## Files Created/Modified
+
+- `internal/build/new_golden_test.go`: `update` flag, `goldenAppKey`, `newGoldenApp`, `goldenName`, `assertGolden` with first-diff output, `TestNewGolden`, `TestNewGoldenNoPlugin`
+- `internal/build/testdata/new/*.golden` (17), `internal/build/testdata/new-no-plugin/*.golden` (6)
+- `internal/build/new_test.go`: helpers (`parentWithFramework`, `fakeFramework`, `stubBinaryFramework`, `isolateGit`) and 16 `TestNew*` tests
+- `internal/build/build_test.go`: relative go.work assertion, `TestMakePluginRoutesUnchanged`
+- `modules/lagoon/keygen_test.go`: `TestGenerateAppKey`
+- `cmd/summer/main_test.go`: `new` help flags, `TestNewCommandViaCLI`
+- `modules/sunrise/script_behavior_test.go`: `scriptHarness`, `fakeRemote`, `TestScriptBehavior`, `TestScriptShellcheck`
+- `scripts/check-phase14.3.sh`: `REQUIRED_TESTS`, `require_named_pass`, `--unit`, `--forbidden`, `--bootstrap`, `--terminal`, `--security`, `--all`
+- `14.3-SECURITY-REVIEW.md`: rows T-14.3-20..26, pins for every row, counts 27/27, 4 accepted
+- `14.3-VALIDATION.md`: `<pkg>` → `sunrise`, exact tests and commands per plan, validated and nyquist_compliant
+
+## Decisions Made
+
+- The goldens exercise sibling auto-detection (no `--framework`), so the golden replace is deterministic and the sibling path is covered.
+- `TestGenerateAppKeyIs32ByteBase64` became `TestGenerateAppKey`, the name the plan and the gate use. The assertions are unchanged.
+- The gate adds named site tests and a goldens-unchanged check beyond the plan's list. Both are cheap and close T-14.3-24 at gate level.
+
+## Deviations from Plan
+
+### Auto-fixed Issues
+
+**1. [Rule 1 - Bug] Detached git auto maintenance raced temp-dir cleanup**
+- **Found during:** Task 2
+- **Issue:** In the full `TestNew` run, `TestNewGitFirstCommit` failed on `TempDir RemoveAll cleanup: unlinkat .../.git: directory not empty`. git 2.55 runs maintenance detached after a commit and wrote `.git/info/refs` and `objects/info/packs` while the directory was being removed.
+- **Fix:** `isolateGit` (and the script harness's git environment) sets `GIT_CONFIG_COUNT` with `maintenance.auto=false` and `gc.auto=0`. This is test-only; the generator itself is unchanged.
+- **Files modified:** internal/build/new_test.go, modules/sunrise/script_behavior_test.go
+- **Verification:** five consecutive `TestNewGit*` runs green, no leftover `/tmp/TestNew*`
+- **Committed in:** 392dd3c, 93e9a8a
+
+**2. [Rule 3 - Blocking] Gate marker literal appeared twice**
+- **Found during:** Task 3 acceptance check (`grep -c 'Phase 14.3 gate passed'` must be 1)
+- **Issue:** The new header comment quoted the marker.
+- **Fix:** Reworded the comment (comment-only change after the `--all` run).
+- **Committed in:** c4c91f0
+
+---
+
+**Total deviations:** 2 auto-fixed (1 bug, 1 blocking). **Impact:** None on scope; the first one only affects how tests clean up.
+
+## Issues Encountered
+
+- `internal/build/registry.go` is still listed by `gofmt -l`. It predates this phase and is out of scope.
+- `shellcheck` on `scripts/check-phase14.3.sh` reports three SC2086 info notes in plan 02's `acc_cleanup`, which relies on word splitting. Left as is.
+
+## User Setup Required
+
+None for the tests. Release steps (D-19, D-28), for the user only. Nothing was pushed, tagged or deployed:
+
+1. Review and push summercms.go `master`.
+2. Tag the framework: `git tag -a v0.1.13 -m "v0.1.13"` on the commit holding all Phase 14.3 framework changes, then `git push origin v0.1.13`. This is one-way: a defect found afterwards ships as v0.1.14.
+3. In `sm-summercmsio-app`, bump `require git.golem15.com/golem15/summercms` to `v0.1.13` in `go.mod` and `plugins/golem15/summercms/go.mod`, and update the `v0.1.12` references in DEPLOY.md and CLAUDE.md. Commit in the plugin submodule and the app, then push the submodules before the app (`ssu`).
+4. Run `scripts/build.sh` (release) and deploy per `sm-summercmsio-app/DEPLOY.md`.
+5. Verify production. `curl -fsSI https://summercms.io/new` should show `200` and `text/plain; charset=utf-8`. Run `SUMMERCMS_TERMINAL_CHECK=1 go test -run TestTerminalCommands -count=1 -v .` in `sm-summercmsio-app` with no overrides, then `bash scripts/check-phase14.3.sh --acceptance-production` in summercms.go.
+
+## Next Phase Readiness
+
+- Phase 14.3 complete, ready for `/gsd-verify-work 14.3` and then the release checklist above.
+
+---
+*Phase: 14.3-one-command-app-scaffolding-summer-new-and-summercms-io-new*
+*Completed: 2026-10-08*
+
+## Self-Check: PASSED