docs(phase-14.3): add validation strategy and post-research decisions
This commit is contained in:
Jakub Zych
@@ -0,0 +1 @@
|
||||
|
||||
@@ -55,6 +55,15 @@ Out of scope: `sm-<name>-theme` (Nuxt/Vue frontend) scaffolding — but nothing
|
||||
- **D-20:** In a clean `node:24-bookworm`-style container with Go 1.27, Docker access, git and jq, starting from nothing: `curl … | bash -s -- demo` → `cd sm-demo-app && docker compose up -d && summer build && summer migrate && summer serve`, then a starter-plugin route answers over HTTP. Before the site is deployed, the run may fetch the script from a local server or the repo file, and clone from a local path/tag via the D-17 overrides; the final run against https://summercms.io/new happens after deploy.
|
||||
- **D-21:** Generator Go tests follow the repo's existing test conventions (stdlib `testing`, golden files under `testdata/`, `-short` gating for slow paths such as the real build). Unit tests are the last plan.
|
||||
|
||||
### Post-research decisions (2026-10-08, researcher recommendations adopted)
|
||||
- **D-22:** On top of the D-02 regex, `summer new` rejects names that break a Go build: `main`, `internal`, and every Go keyword (e.g. `go`, `func`, `type`). Error text says why. — **Reversibility:** reversible — loosening later is non-breaking.
|
||||
- **D-23:** `compose.yaml` publishes Postgres on a fixed non-5432 host port bound to `127.0.0.1` (planner picks the number); `.env`/`.env.example` DSN, user, password and DB name match it.
|
||||
- **D-24:** `summer new` prints the next steps itself (so a manual `summer new` user sees them too); the bootstrap script prints only the extra bits (the `cd` line, the PATH hint).
|
||||
- **D-25:** No root `go.sum` in the scaffold; building goes through `go.work`. `go.work.sum` is gitignored.
|
||||
- **D-26:** The new exported framework package that embeds the bootstrap script lives under `modules/` (name is planner's call, it must follow the module README rules and get a root README table row). The script file sits inside that package directory (`//go:embed` cannot reach `../scripts/`). Script env overrides are `SUMMERCMS_REPO`, `SUMMERCMS_DIR`, `SUMMERCMS_REF` — never `SUMMER_*` (compass reads those as config).
|
||||
- **D-27:** Fix the two defects research found, in the shared code paths: `plugin:add` writes a relative path into `go.work` (not absolute), and the `make:plugin` routes stub gains an opt-in starter route used by `summer new` (no duplicated template). Export `lagoon.GenerateAppKey()` (or equivalent) so the key is generated without the `key:generate` CLI's `✓ ` prefix. Add `config/http.yaml` to `examples/hello` so `TestTypedItemRoute` passes.
|
||||
- **D-28:** Release ordering: all framework work lands before tagging `v0.1.13`. The site plugin's `/new` route and the landing-page change are built against the framework through the site's local `go.work`/replace during this phase; bumping the site's `require` to `v0.1.13` and redeploying happen after the tag (user action, per `sm-summercmsio-app/DEPLOY.md`). The bootstrap script's `SUMMERCMS_REF` override lets acceptance run before the tag exists.
|
||||
|
||||
### Claude's Discretion
|
||||
- Exact package layout inside `internal/build` (e.g. `new.go` + templates under `internal/build/stubs/new/`), flag names beyond those listed, and template engine (`text/template` like existing stubs).
|
||||
- Whether `summer new` also runs `summer build` at the end (default: no — the next-steps text covers it).
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
phase: "14.3"
|
||||
slug: "one-command-app-scaffolding-summer-new-and-summercms-io-new"
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: "2026-10-08"
|
||||
---
|
||||
|
||||
# Phase 14.3 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution. Source: 14.3-RESEARCH.md § Validation Architecture.
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | Go stdlib `testing` (Go 1.27.0); `node:test` in `vue-summercmsio-app` |
|
||||
| **Config file** | none |
|
||||
| **Quick run command** | `go test ./internal/build/ ./cmd/summer/ ./modules/lagoon/ ./modules/<pkg>/ -short` |
|
||||
| **Full suite command** | `go vet ./... && go test ./... && go -C examples/hello test ./...` |
|
||||
| **Estimated runtime** | ~40 seconds quick (incl. real builds), a few minutes full |
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** quick run command
|
||||
- **After every plan wave:** full suite + the plan's sibling-repo command (below)
|
||||
- **Before `/gsd-verify-work` and before tagging `v0.1.13`:** full suite green, `go run ./cmd/summer docs:build --check` green, D-20 acceptance run recorded
|
||||
- **Max feedback latency:** ~60 seconds
|
||||
|
||||
---
|
||||
|
||||
## Per-Plan Verification Map
|
||||
|
||||
| Plan | Behavior | Test Type | Automated Command |
|
||||
|------|----------|-----------|-------------------|
|
||||
| 01 | `summer new demo` in a temp parent with a `summercms` symlink to the repo builds | smoke (real build) | `go test ./internal/build/ -run 'TestNewApp' -count=1` |
|
||||
| 01 | `summer new` wired in CLI with its flags | unit | `go test ./cmd/summer/ -run 'TestToolCommandNames' -count=1` |
|
||||
| 01 | `GenerateAppKey` returns StdEncoding of 32 random bytes; `key:generate` uses it | unit | `go test ./modules/lagoon/ -run 'Key' -short -count=1` |
|
||||
| 01 | `plugin:add` writes relative `go.work` paths | unit | `go test ./internal/build/ -run 'TestMakeAndAddPluginOnHelloCopy' -count=1` |
|
||||
| 01 | `examples/hello` config gap fixed | integration | `go -C examples/hello test ./... -count=1` |
|
||||
| 02 | script: shebang, `main "$@"` last line, `bash -n`, truncation runs nothing, tag selection on a temp repo | unit + gated e2e | `go test ./modules/<pkg>/ -count=1`; gated `SUMMERCMS_BOOTSTRAP_CHECK=1 go test ./modules/<pkg>/ -run TestBootstrapEndToEnd -count=1` |
|
||||
| 02 | `GET /new` 200 `text/plain; charset=utf-8`, body = embedded script, HEAD ok, POST 405, nosniff | unit | `go -C ../sm-summercmsio-app/plugins/golem15/summercms test ./... -count=1` |
|
||||
| 02 | D-20 container chain ends with starter route 200 | gated/manual | acceptance runner script or documented `docker run` command |
|
||||
| 03 | docs tree valid, `summer new` known command, llms-full contains one-liner | unit | `go test ./cmd/summer -run 'TestDocsTree|TestDocsBuildRealTree' -count=1` and `go run ./cmd/summer docs:build --check` |
|
||||
| 03 | landing page terminal data / tests | unit | `go -C ../sm-summercmsio-app test -run 'TestLoadTerminal|TestTerminalScript|TestTerminalEnv' .`; `pnpm -C ../sm-summercmsio-app/vue-summercmsio-app generate && pnpm -C ../sm-summercmsio-app/vue-summercmsio-app test` |
|
||||
| 04 | golden output for every rendered file; name/denylist/existing-dir/framework-detection errors; relative replace; `--no-plugin`, `--no-git`; no-identity commit; cleanup on failure; real build (`-short` skip) | unit + golden | `go test ./internal/build/ -run 'TestNew' -count=1`; update goldens with `-update` |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
- [ ] `internal/build/testdata/new/` golden files + `-update` flag (no golden infra exists yet)
|
||||
- [ ] new `modules/<pkg>/` package with README and tests
|
||||
- [ ] test helper creating `<tmp>/summercms` → repo-root symlink so the generated replace is deterministic (`../summercms`)
|
||||
- [ ] optional acceptance runner script
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Why Manual | Test Instructions |
|
||||
|----------|------------|-------------------|
|
||||
| `curl -fsSL https://summercms.io/new \| bash -s -- demo` against production | needs `v0.1.13` tag pushed and the site redeployed (user actions) | run D-20 chain in a clean `node:24-bookworm` container with Go 1.27 and the host Docker socket |
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
|
||||
- [ ] Wave 0 covers all MISSING references
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 60s
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
Reference in New Issue
Block a user