From b91eadb5b67592dd6cc7823860c0cf4b23d9afbf Mon Sep 17 00:00:00 2001 From: Jakub Zych Date: Thu, 29 Oct 2026 18:47:57 +0100 Subject: [PATCH] docs(34-04): complete plugin:install and plugin:update plan --- .../34-plugin-marketplace/34-04-SUMMARY.md | 255 ++++++++++++++++++ 1 file changed, 255 insertions(+) create mode 100644 .planning/phases/34-plugin-marketplace/34-04-SUMMARY.md diff --git a/.planning/phases/34-plugin-marketplace/34-04-SUMMARY.md b/.planning/phases/34-plugin-marketplace/34-04-SUMMARY.md new file mode 100644 index 00000000..f8e122ce --- /dev/null +++ b/.planning/phases/34-plugin-marketplace/34-04-SUMMARY.md @@ -0,0 +1,255 @@ +--- +phase: 34-plugin-marketplace +plan: 04 +subsystem: plugins +tags: [kiosk, fetchguard, plugin-install, plugin-update, go-get, git-submodule, go-work, registry-client] + +requires: + - phase: 34-plugin-marketplace (34-01) + provides: "kiosk wire types and version rules (RegistryAPIPath, RegistryVersion, ValidVersion, CompareVersions, BaseVersion, Satisfies, NewestCompatible), kiosk.Parse, kiosk.ReadIdentity, build.EffectiveFramework" + - phase: 34-plugin-marketplace (34-03) + provides: "the registry server at /api/registry/v1 on summercms.io" +provides: + - "kiosk.Client (List, Plugin, Versions, BaseURL) over fetchguard TrustedMode, 4 MiB, 10 s, no redirects; kiosk.NewClient, kiosk.RegistryURL, kiosk.DefaultRegistryURL, kiosk.ErrRegistryDisabled, kiosk.ErrNotFound; config key plugins.registry_url" + - "build.PlanInstall / build.ApplyInstall with InstallOptions, InstallPlan, InstallStep, Registry, ErrCancelled, ModeModule, ModeSubmodule" + - "build.Update with UpdateOptions" + - "summer plugin:install [--source] [--yes] [--offline]; summer plugin:update [id] [--yes] [--offline]" + - "comment-preserving summer.yaml appends (appendManifestPlugins) used by plugin:add and plugin:install" + - "go.work replace overrides for submodule plugins (refreshWorkOverrides)" +affects: [34-05 admin Plugins screen (kiosk.Client, plugins.registry_url), 34-06 tests and release ledger, every app that installs plugins] + +actuals: + tokens: 39000 + tasks: 4 + commits: 5 +plan_head_before: e7eda88ddd8ffb5f5626bc6af4a7c5aa765e1e75 +plan_head_after: 96a7012ece88c0e0a095b84b7d1b3c4ee31f8898 + +tech-stack: + added: [] + patterns: + - "Plan, print, confirm, apply: PlanInstall/Update resolve and print everything before any change; --yes or an interactive yes is required (never bonfire.ConfirmProceed)" + - "Every registry string is validated before exec: strict vX.Y.Z, plain module paths, package inside module, https repository URLs; git runs with GIT_ALLOW_PROTOCOL and -- before URLs and paths" + - "Offline Go module tests: file:// GOPROXY fixture tree built with archive/zip, chained to the developer's module download cache, with a throwaway GOMODCACHE" + +key-files: + created: + - modules/kiosk/client.go + - internal/build/install.go + - internal/build/source.go + - internal/build/update.go + - internal/build/install_smoke_test.go + - internal/build/fixtures_test.go + modified: + - internal/build/manifest.go + - internal/build/scaffold.go + - cmd/summer/main.go + - modules/kiosk/README.md + - docs/plugins/marketplace.md + - docs/console/scaffolding.md + - docs/architecture/go-modules-and-workspaces.md + - .planning/research/questions.md + +key-decisions: + - "A workspace module that another workspace module's go.mod replaces gets a versioned go.work replace (M@ => dir): Go refuses a whole-path replace of a workspace module, and any go.work replace of the path resolves the conflict (verified in scratch)" + - "go.work overrides are computed over every go.work use module's go.mod, not only the newly installed plugin, so a plugin installed earlier that replaces the new one is covered; replaces the user wrote in go.work are left alone; self-pins (M => M@v) follow the app's go.mod require" + - "plugin:update reads each plugin's source and version from go.mod/go.work and git describe (installedSource), offline, instead of the go list resolver, which needs every module's sources downloaded" + - "A git URL install with no fitting release tag takes the default branch head with a warning and requires v0.0.0 in go.mod, as plugin:add does" + - "plugin:update without an ID applies what resolved, reports the rest under 'Not updated', and fails only when a resolved update could not be carried out" + - "--module for git URLs is not in v1 (D-11); the research question is closed" + +patterns-established: + - "Commands that change an application's plugins print every change as id@version (mode) and need --yes off a terminal" + - "Install and update tests use fakeRegistry (httptest + the real kiosk.Client), writeProxyModule and pluginRepo fixtures, and never touch the network" + +requirements-completed: [SC-3, D-08, D-09, D-10, D-11, D-12, D-18] + +coverage: + - id: D1 + description: "summer plugin:install installs the newest compatible registry release as a Go module with go get, appends summer.yaml in place (comments kept) and the app builds; a pin needing a newer framework fails naming it" + requirement: SC-3 + verification: + - kind: integration + ref: "internal/build/install_smoke_test.go#TestInstallSmoke/module" + status: pass + - kind: unit + ref: "internal/build/install_smoke_test.go#TestInstallSmoke/module_pin_framework,append_manifest" + status: pass + human_judgment: false + - id: D2 + description: "Transitive requirements, installed-too-old error naming summer plugin:update, cycle report, pins, unknown IDs, prompt and --yes rule, --offline/disabled registry, refusal of invalid registry strings" + requirement: D-09 + verification: + - kind: unit + ref: "internal/build/install_smoke_test.go#TestInstallSmoke/deps,installed_old,cycle,pin,unknown,prompt,offline,invalid" + status: pass + human_judgment: false + - id: D3 + description: "--source and git URL installs as git submodules at the chosen tag, registered with the tag as the go.mod require, with go.work overrides so the app builds although the plugin replaces the framework with ../summercms.go; non-git apps refused; URL validation" + requirement: D-12 + verification: + - kind: integration + ref: "internal/build/install_smoke_test.go#TestInstallSmoke/source,source_deps,giturl,giturl_offline" + status: pass + - kind: unit + ref: "internal/build/install_smoke_test.go#TestInstallSmoke/nogit,urls" + status: pass + human_judgment: false + - id: D4 + description: "summer plugin:update for module and submodule installs, all-plugins mode with skipped/failed reporting, go.work pin refresh, prompt rule" + requirement: D-10 + verification: + - kind: integration + ref: "internal/build/install_smoke_test.go#TestInstallSmoke/update,update_submodule,update_all,update_prompt" + status: pass + human_judgment: false + - id: D5 + description: "kiosk.Client and plugins.registry_url (default, override, empty disables), fetchguard limits, 404 as ErrNotFound naming the registry" + requirement: D-18 + verification: + - kind: unit + ref: "internal/build/install_smoke_test.go#TestInstallSmoke/registry_client,unknown,offline" + status: pass + human_judgment: false + - id: D6 + description: "CLI commands and docs: plugin:install and plugin:update documented, kiosk README, marketplace, scaffolding and go-modules pages; --module question closed" + requirement: D-11 + verification: + - kind: other + ref: "go test ./cmd/summer -run 'TestCommandsDocumented|TestDocsTree'; go run ./cmd/summer docs:build --check" + status: pass + human_judgment: true + rationale: "Docs and help wording are a reviewer's call; the checker only proves identifiers, command names, links and snippets" + - id: D7 + description: "Interactive prompt on a real terminal and installs against the live summercms.io registry and real plugin repositories over https/ssh" + verification: [] + human_judgment: true + rationale: "Tests run non-interactively against httptest and file:// fixtures; a real terminal prompt, credentials and the deployed registry need a person" + +duration: 34min +completed: 2026-10-29 +status: complete +--- + +# Phase 34 Plan 04: plugin:install and plugin:update Summary + +**`summer plugin:install` adds registry plugins as Go modules (newest release the app's effective framework allows, with their requirements, after a printed plan and a confirmation or `--yes`), installs from source or any git URL as git submodules with the go.work overrides that make them build, and `summer plugin:update` moves module and submodule installs to their newest compatible release; all registry calls go through the new `kiosk.Client` and the `plugins.registry_url` setting.** + +## Performance + +- **Duration:** about 34 min +- **Started:** 2026-10-29T17:12:51Z +- **Completed:** 2026-10-29T17:46:30Z +- **Tasks:** 4 (Task 1 tracer, then three expansion tasks) +- **Files modified:** 14 (6 created, 8 modified) + +## Accomplishments + +- **Registry client (D-18).** `kiosk.Client` (`List`, `Plugin`, `Versions`) over a `fetchguard` client in trusted mode with a 4 MiB cap, a 10 s timeout and no redirects. `kiosk.RegistryURL` reads `plugins.registry_url` (default `https://summercms.io`, `SUMMER_PLUGINS__REGISTRY_URL` overrides, empty disables). A 404 wraps `kiosk.ErrNotFound` and names the ID and the registry origin; a redirect is an error asking for the final origin. +- **Module installs (D-08).** `build.PlanInstall` parses `id[@vX.Y.Z]`, asks the registry, picks the newest release the effective framework satisfies (or the pin), and fails with the framework version to move to when only newer-framework releases exist. `build.ApplyInstall` runs one `go get -- mod@v ...` and appends summer.yaml in place. It does not run `go mod tidy` (Pitfall 5). +- **Requirements, pins and prompts (D-09).** Requirements resolve transitively, dependencies first, each at the newest compatible release at or above its minimum. Plugins sharing a module share one release. An installed requirement below its minimum is an error naming `summer plugin:update `. Cycles are reported. The plan is printed as `id@version (mode)`; `--yes` skips the prompt, a non-interactive run without it is an error naming `--yes`, and a declined prompt changes nothing (`build.ErrCancelled`). +- **Submodule installs (D-09, D-11, D-12, research F2).** `--source` clones each registry repository (https only) at the chosen tag. A git URL (https, ssh, `user@host:path`) is cloned with the user's own git setup and never sent to the registry. The command reads `plugin.yaml` at the newest tag whose framework minimum fits, or falls back to the default branch head with a warning. Each plugin is moved to `plugins//`, then `git submodule add` and `absorbgitdirs` run, and the plugin is registered with the tag as its go.mod require (plugin:add keeps `v0.0.0`). The checkout's `plugin.yaml` must match the listed ID and module. `refreshWorkOverrides` adds the go.work replaces the plugins' development-layout `go.mod` replaces need. A non-git app is refused before cloning. +- **plugin:update (D-10).** Module installs move with one `go get`. Submodules fetch tags, are checked out at the release tag and require it in go.mod. Unlisted submodules (git URL installs) use their own tags and `plugin.yaml`. Without an ID, every resolvable plugin is updated and the rest are reported. A newer release that needs a newer framework is named. +- **summer.yaml edits keep comments** (user decision 5). `appendManifestPlugins` inserts after the last item of the `plugins:` block at its indentation, re-parses, and restores the original bytes on any error. `plugin:add` uses it too. +- `--module` research question closed as "not in v1" (D-11). + +## Task Commits + +1. **Task 1: plugin:install for registry plugins as Go modules (tracer)**: `9b23e87` (feat). Tracer gate: interactive, `end-of-phase`, automated-only verify, re-run green, expanded. +2. **Task 2: requirements, pins, prompts and offline rules**: `a462bf1` (feat) +3. **Task 3: --source and git URLs as submodules with go.work overrides**: `8093587` (feat) +4. **Task 4: summer plugin:update**: `d024ab4` (feat); **--module question closed**: `96a7012` (docs, planning) + +**Plan metadata:** the docs commit containing this SUMMARY. + +## Offline module-install test environment (as requested by the plan) + +`TestInstallSmoke/module`, `deps` and `update` ran with: + +- `GOPROXY=file://,file:///cache/download` +- `GOMODCACHE=` +- `GOSUMDB=off`, `GOTOOLCHAIN=local`, with `GONOSUMDB`, `GOPRIVATE` and `GONOPROXY` empty +- `GOFLAGS=-modcacherw` set in the test env. summer's go runner (`applyGoWorkFile`) drops `GOFLAGS`, and for an app without go.work it sets `GOWORK=off GOFLAGS=-mod=mod`. A test cleanup therefore makes the fresh module cache writable before the temp dir is removed. + +The fixture proxy tree has `@v/list`, `.info`, `.mod` and `.zip` files, with zip entries prefixed `@/`. No network was used. `module` takes about 28 s, mostly because the fresh module cache extracts every dependency before `summer build` links the app. The submodule tests run with `GOPROXY=off`. + +## Files Created/Modified + +- `modules/kiosk/client.go`: registry client and `plugins.registry_url` +- `internal/build/install.go`: `PlanInstall`, `ApplyInstall`, resolution, validation, prompt rule, `installedVersion` +- `internal/build/source.go`: CLI git runner (`cliGitProtocols`), `isGitURL`, `validateGitURL`, `validateRepositoryURL`, clone, tag and `plugin.yaml` selection, `applySubmodule`, `refreshWorkOverrides` +- `internal/build/update.go`: `Update`, `installedSource` +- `internal/build/manifest.go`: `appendManifestPlugins`, `insertManifestPlugins` +- `internal/build/scaffold.go`: `AddPlugin` appends in place; `editAppModule` takes the require version +- `internal/build/install_smoke_test.go`, `internal/build/fixtures_test.go`: TestInstallSmoke (module, module_pin_framework, deps, installed_old, cycle, pin, unknown, prompt, offline, invalid, source, source_deps, giturl, giturl_offline, nogit, urls, update, update_submodule, update_all, update_prompt, append_manifest, registry_client) and the helpers writeProxyModule, fakeRegistry, helloCopy, pluginRepo, gitApp +- `cmd/summer/main.go`: `plugin:install`, `plugin:update`, `registryFor` +- Docs: `modules/kiosk/README.md` (client, Configuration, CLI rows), `docs/plugins/marketplace.md` (Installing plugins, Required plugins, Confirmation, Installing from source, Installing from a git URL, Updating plugins, The registry setting), `docs/console/scaffolding.md` (two rows), `docs/architecture/go-modules-and-workspaces.md` (Plugins installed as submodules) +- `.planning/research/questions.md`: the `--module` entry answered + +## Decisions Made + +See `key-decisions` in the frontmatter. In short: versioned go.work replaces for workspace modules, overrides computed across all workspace modules, offline source detection for update, a default-branch-head fallback that requires `v0.0.0`, and lenient update-all semantics (apply what resolved, report the rest). + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] Workspace modules need a versioned go.work replace** +- **Found during:** Task 3 +- **Issue:** The plan's F2 rule replaces a module that is another installed submodule with "the workspace directory of that module". Go rejects that: "workspace module X is replaced at all versions in the go.work file". Without some go.work replace, it reports "conflicting replacements" instead. +- **Fix:** `refreshWorkOverrides` writes `replace M => ./plugins/...` for workspace modules. This was verified in a scratch workspace, where any go.work replace of the path resolves the conflict, even when MVS selects another version. +- **Files modified:** internal/build/source.go +- **Commit:** 8093587 + +**2. [Rule 2 - Missing critical] Overrides cover all workspace modules, not just the new plugin** +- **Found during:** Task 3 +- **Issue:** A submodule installed earlier whose go.mod replaces the newly installed module breaks the build the same way. +- **Fix:** Every go.work `use` module's go.mod is scanned (except the app's). The function is idempotent, and replaces already in go.work are never touched. +- **Commit:** 8093587 + +**3. [Rule 2 - Missing critical] Extra URL hardening** +- Git URLs carrying a password, and registry repository URLs carrying any user info, are refused, so credentials never land in `.gitmodules`. `file://` registry repositories are accepted only when `cliGitProtocols` includes `file` (the tests). A target directory that already exists is refused while planning (registry `--source`) or before cloning (git URL). +- **Commits:** 8093587 + +**4. [Rule 3 - Blocking] The docs checker refuses names of commands that do not exist yet** +- Tasks 1-3 could not name `summer plugin:update` in docs (`docs:build --check`: "plugin:update is not a summer or application command"). Those sentences were reworded and then restored in Task 4. Because of that, Task 4 also touched `modules/kiosk/README.md` and `docs/architecture/go-modules-and-workspaces.md`, which are not in its file list. The install error message names `summer plugin:update` from Task 2 on, as the acceptance criteria require. +- **Commit:** d024ab4 + +**5. [Plan adjustment] Source and version detection for update** +- The plan says to use "the 34-01 resolver" (go list). `installedSource` reads go.mod, go.work and `git describe` instead, applying the D-07 rules: module version, base tag for a plugin that is its own repository, nothing for an in-app plugin. It needs no downloads, and the installed-version check in plugin:install uses it too. +- **Commit:** d024ab4 + +**6. [Plan adjustment] Extra exported names** +- `build.ErrCancelled`, `build.ModeModule`, `build.ModeSubmodule` and `kiosk.Client.BaseURL` are not in the plan's artifact list. They give the CLI a declined-prompt path and give 34-05 the registry origin. `kiosk.Client.BaseURL` is documented in the kiosk README. + +**7. [Plan adjustment] Update reports newer incompatible releases next to an update** +- `acme.translate v0.1.0 -> v0.1.1 (submodule); v0.2.0 needs framework v9.0.0`, so the admin badge and the CLI tell the same story. + +**8. [Process] TDD order** +- Task 2: the behavior subtests were written first and failed on their assertions (deps, installed_old, cycle, prompt and invalid) before the implementation. Task 4: the subtests were written first and failed to compile on the missing `Update`/`UpdateOptions`. Task 3: the implementation came before its subtests. As a substitute RED check, `refreshWorkOverrides` was disabled and `source`/`source_deps` were run against it; both failed on the go.work assertions, and the code was then restored. Each task is one commit, as the plan's Execution units table requires, so there are no separate `test(...)` commits. + +--- + +**Total deviations:** 8 (1 Rule 1, 2 Rule 2, 1 Rule 3, 3 plan adjustments, 1 process note). **Impact:** no contract change. All planned names exist, `plugin:add` behaviour is unchanged (TestMakeAndAddPluginOnHelloCopy passes), and the overrides follow verified Go behaviour. + +## Issues Encountered + +- `gofmt -l cmd/summer` lists pre-existing alignment differences in `cmd/summer/main.go` (`Args: makeArtifactArgs()` lines), `runtime.go` and `runtime_test.go`, plus `modules/kiosk/manifest_smoke_test.go`. This plan did not introduce them (scope boundary). The new code is gofmt-clean. +- A plugin repository that has only nested plugins and no root `plugin.yaml` or `plugin.go` cannot be installed from a git URL; the command reports this. Registry `--source` installs of nested plugins do work: the checkout directory is named after the root plugin, or after the repository name without `sm-` and `-plugin`. + +## User Setup Required + +None. `plugins.registry_url` defaults to https://summercms.io. + +## Next Phase Readiness + +- 34-05 (admin Plugins screen) can build its registry client with `kiosk.NewClient(kiosk.RegistryURL(app.Config), app.Config)`, treat `kiosk.ErrRegistryDisabled` as "hide the Marketplace tab", and compute "update available" with the same rule as `plugin:update` (`kiosk.NewestCompatible` above the installed version). +- 34-06: TestInstallSmoke is the install/update coverage. The non-short subtests take about 45 s, most of it in `module`. +- Real-world check still open (D7): run `summer plugin:install golem15.` against the deployed registry, and run a git URL install over SSH, on a terminal. + +## Self-Check: PASSED + +- Key files exist: modules/kiosk/client.go (`DefaultRegistryURL = "https://summercms.io"`, `func RegistryURL(`), internal/build/install.go (`func PlanInstall(`, `NewestCompatible`), internal/build/source.go (`"work", "edit"`, `absorbgitdirs`, `"https:ssh"`), internal/build/update.go (`func Update(ctx context.Context, opts UpdateOptions) error`) and internal/build/install_smoke_test.go (`func TestInstallSmoke(`). +- Commits 9b23e87, a462bf1, 8093587, d024ab4 and 96a7012 are in history; `git rev-list --count e7eda88..HEAD` = 5 before this docs commit. +- `go vet ./...`, `go test -short ./...`, `go test ./internal/build ./cmd/summer ./modules/kiosk` (non-short, all TestInstallSmoke subtests), `cd examples/hello && go test ./...`, `go test ./cmd/summer -run 'TestCommandsDocumented|TestDocsTree'` and `summer docs:build --check` all pass. +- Acceptance criteria of Tasks 1-4 were re-run and pass. `cmd/summer/main.go` and `internal/build/install.go` do not call `ConfirmProceed`, and `grep -i "not in v1" .planning/research/questions.md` hits the `--module` entry.