feat(build): summer plugin:install for registry plugins as Go modules
- kiosk.Client reads the registry API v1 over fetchguard (trusted mode, 4 MiB, 10 s, no redirects); kiosk.RegistryURL reads plugins.registry_url (default https://summercms.io, empty disables) - build.PlanInstall/ApplyInstall pick the newest release the effective framework satisfies (or a pin), validate registry strings, go get it and append it to summer.yaml - summer.yaml edits (plugin:add and plugin:install) keep comments and blank lines - TestInstallSmoke/module installs from a file:// GOPROXY and builds
This commit is contained in:
@@ -21,6 +21,7 @@ WinterCMS keeps this data in a plugin's `Plugin.php` `pluginDetails()` and in `c
|
||||
- Name and description overrides (`--name`, `--description`) stored in an `overrides:` block that regeneration keeps.
|
||||
- The registry API v1 contract in Go: `kiosk.RegistryAPIPath` and the wire types (`kiosk.ListResponse`, `kiosk.DetailResponse`, `kiosk.VersionsResponse`, `kiosk.ErrorResponse`, `kiosk.IngestRequest`, `kiosk.IngestResponse`). The registry server encodes with them and the summer tool and the admin decode with them, so the JSON shape exists once.
|
||||
- Strict version rules: `kiosk.ValidVersion` (only `vX.Y.Z`), `kiosk.CompareVersions`, `kiosk.BaseVersion` for `git describe` output, `kiosk.Satisfies` for Go-style minimums and `kiosk.NewestCompatible` to pick the release an application can install.
|
||||
- A registry client, `kiosk.Client`, for the summer tool and the admin: `kiosk.NewClient` builds it over a `fetchguard` client in trusted mode with a 4 MiB body cap, a 10 second timeout and no redirects, and `kiosk.RegistryURL` reads the registry origin from the `plugins.registry_url` config key (default `kiosk.DefaultRegistryURL`; an empty value disables registry calls).
|
||||
- `kiosk.Check` regenerates in memory and returns a `kiosk.StaleError` when the committed file is missing or differs, which `summer plugin:manifest --check` turns into exit status 1, and `kiosk.CheckT` turns into a failing test in the plugin's own `go test`.
|
||||
|
||||
## Usage
|
||||
@@ -58,6 +59,22 @@ if !ok {
|
||||
fmt.Println("go get", v.Module+"@"+v.Version)
|
||||
```
|
||||
|
||||
Asking the configured registry for a plugin's releases:
|
||||
|
||||
```go
|
||||
client, err := kiosk.NewClient(kiosk.RegistryURL(app.Config), app.Config)
|
||||
if errors.Is(err, kiosk.ErrRegistryDisabled) {
|
||||
return nil // plugins.registry_url is empty: no registry calls
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
versions, err := client.Versions(ctx, "acme.blog")
|
||||
if errors.Is(err, kiosk.ErrNotFound) {
|
||||
return fmt.Errorf("acme.blog is not listed")
|
||||
}
|
||||
```
|
||||
|
||||
A plugin repository keeps its manifest current with one test:
|
||||
|
||||
```go
|
||||
@@ -111,21 +128,38 @@ func TestPluginManifest(t *testing.T) {
|
||||
| `kiosk.BaseVersion` | The `vX.Y.Z` tag a `git describe --tags` string starts from (`v0.3.1` for `v0.3.1-4-gabc1234`), or `""`. |
|
||||
| `kiosk.Satisfies` | Whether a release meets a minimum; an empty minimum is always met, and a pseudo-version minimum is met by its base release and later. |
|
||||
| `kiosk.NewestCompatible` | The newest release whose framework minimum a framework version satisfies. |
|
||||
| `kiosk.DefaultRegistryURL` | `https://summercms.io`, the registry used when `plugins.registry_url` is not set. |
|
||||
| `kiosk.RegistryURL` | The registry origin from a `compass.Config`: the `plugins.registry_url` value without a trailing slash, `kiosk.DefaultRegistryURL` when the key is absent, `""` (disabled) when it is set empty. |
|
||||
| `kiosk.ErrRegistryDisabled` | Returned by `kiosk.NewClient` for an empty base URL. |
|
||||
| `kiosk.ErrNotFound` | Wrapped by a `kiosk.Client` error when the registry answers 404; the message names the ID and the registry origin. |
|
||||
| `kiosk.Client` | Calls the registry API v1 of one origin through `fetchguard` (trusted mode, 4 MiB cap, 10 s timeout, no redirects). |
|
||||
| `kiosk.NewClient` | Builds a client for an `http` or `https` origin without user info, query or fragment; the config supplies fetchguard defaults and may be nil. |
|
||||
| `kiosk.Client.BaseURL` | The origin the client calls. |
|
||||
| `kiosk.Client.List` | `GET /plugins?q=&category=`, decoded as a `kiosk.ListResponse`. |
|
||||
| `kiosk.Client.Plugin` | `GET /plugins/{id}`: one `kiosk.RegistryPluginDetail`. |
|
||||
| `kiosk.Client.Versions` | `GET /plugins/{id}/versions`: the plugin's `kiosk.RegistryVersion` list, newest first. A non-2xx answer other than 404 is an error carrying the registry's error code; a redirect is an error asking for the final origin. |
|
||||
| `kiosk.Identity` | The plugin ID and the required plugin IDs. |
|
||||
| `kiosk.ReadIdentity` | Reads what `ID()` and `Requires()` return from the sources of a directory, resolving constants across packages and modules without running code. Any other expression is an error naming the method. |
|
||||
| `kiosk.StaleError` | `Path`, `Missing`, and the generated (`Want`) and committed (`Got`) bytes; its message names the first differing line. |
|
||||
|
||||
## Configuration
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|-----|---------|---------|
|
||||
| `plugins.registry_url` | `https://summercms.io` | Registry origin for `summer plugin:install` and the admin's Plugins screen (file `config/plugins.yaml`, env `SUMMER_PLUGINS__REGISTRY_URL`). It must be the final origin, because redirects are not followed. An empty value disables registry calls. |
|
||||
|
||||
## CLI commands
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `summer plugin:manifest [dir]` | Writes `plugin.yaml` for the plugin in `dir` (default: the current directory). `--pretend` prints it instead, `--check` fails when the committed file is missing or stale, `--name` and `--description` set or (with an empty value) clear the overrides. |
|
||||
| `summer plugin:install <id\|git-url>` | Installs a plugin; reads its releases through `kiosk.Client` and picks one with `kiosk.NewestCompatible`. See [Installing plugins](../../docs/plugins/marketplace.md#installing-plugins). |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: none.
|
||||
- SummerCMS modules: `compass` (the `plugins.registry_url` setting), `fetchguard` (registry HTTP calls).
|
||||
- Third-party: `github.com/goccy/go-yaml` (manifest YAML), `github.com/yuin/goldmark` (README headings and paragraphs).
|
||||
- Standard library: `bytes`, `context`, `encoding/json`, `errors`, `fmt`, `go/ast`, `go/parser`, `go/token`, `go/types` (expression printing only), `os`, `os/exec`, `path/filepath`, `regexp`, `slices`, `sort`, `strconv`, `strings`, `time`.
|
||||
- Standard library: `bytes`, `context`, `encoding/json`, `errors`, `fmt`, `go/ast`, `go/parser`, `go/token`, `go/types` (expression printing only), `net/http`, `net/url`, `os`, `os/exec`, `path/filepath`, `regexp`, `slices`, `sort`, `strconv`, `strings`, `time`.
|
||||
- Tools: the `go` command (`go mod edit -json`, `go list -m -json`, and `go mod download -json` for a required module not in the module cache), run without a shell and with workspaces off.
|
||||
|
||||
## Testing
|
||||
@@ -135,3 +169,5 @@ go test ./modules/kiosk/...
|
||||
```
|
||||
|
||||
`TestManifestSmoke` copies the fixture plugins in `testdata/` to a temporary directory and generates, checks and edits their manifests: a literal-ID plugin, a plugin whose identity is spread over constants of three packages and two modules, a nested plugin, the error cases, overrides, `kiosk.CheckT` and `kiosk.Validate`, the registry JSON (golden bytes, unknown fields ignored) and the version rules. Every fixture module replaces the framework with the stub in `testdata/framework`, so the tests run the local `go` command but need no network and no database.
|
||||
|
||||
The registry client (`kiosk.Client`) is exercised by `TestInstallSmoke` in `internal/build`, against an `httptest` registry that serves these wire types.
|
||||
|
||||
161
modules/kiosk/client.go
Normal file
161
modules/kiosk/client.go
Normal file
@@ -0,0 +1,161 @@
|
||||
package kiosk
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/compass"
|
||||
"git.golem15.com/golem15/summercms/modules/fetchguard"
|
||||
)
|
||||
|
||||
// DefaultRegistryURL is the registry an application uses when its config
|
||||
// has no plugins.registry_url.
|
||||
const DefaultRegistryURL = "https://summercms.io"
|
||||
|
||||
// registryURLKey is the config key naming the registry origin: the file
|
||||
// config/plugins.yaml, key registry_url, or SUMMER_PLUGINS__REGISTRY_URL.
|
||||
const registryURLKey = "plugins.registry_url"
|
||||
|
||||
// Limits of every registry call: a registry answer is small JSON, so a
|
||||
// larger body or a slow server is an error, not something to wait for.
|
||||
const (
|
||||
registryMaxBytes = 4 << 20
|
||||
registryTimeout = 10 * time.Second
|
||||
)
|
||||
|
||||
var (
|
||||
// ErrRegistryDisabled is returned by NewClient for an empty base URL:
|
||||
// plugins.registry_url is set to "" and registry calls are off.
|
||||
ErrRegistryDisabled = errors.New("the plugin registry is disabled (plugins.registry_url is empty)")
|
||||
// ErrNotFound is wrapped by the errors of a Client call the registry
|
||||
// answered with 404: no plugin has that ID.
|
||||
ErrNotFound = errors.New("not in the registry")
|
||||
)
|
||||
|
||||
// RegistryURL returns the registry origin configured in cfg: the
|
||||
// DefaultRegistryURL when plugins.registry_url is absent (or cfg is nil),
|
||||
// the configured value without a trailing slash otherwise. An empty
|
||||
// result means registry calls are disabled.
|
||||
func RegistryURL(cfg *compass.Config) string {
|
||||
if cfg == nil || !cfg.Has(registryURLKey) {
|
||||
return DefaultRegistryURL
|
||||
}
|
||||
return strings.TrimRight(strings.TrimSpace(cfg.String(registryURLKey)), "/")
|
||||
}
|
||||
|
||||
// Client reads the registry API v1 of one registry origin. Every request
|
||||
// goes through a fetchguard client in TrustedMode (the origin is operator
|
||||
// configuration) with a 4 MiB body cap, a 10 second timeout and no
|
||||
// redirects, so the base URL must be the final origin.
|
||||
type Client struct {
|
||||
base string
|
||||
http *fetchguard.Client
|
||||
}
|
||||
|
||||
// NewClient builds a Client for the registry at baseURL, an http or https
|
||||
// origin without user info, query or fragment, for example
|
||||
// https://summercms.io. An empty baseURL returns ErrRegistryDisabled. cfg
|
||||
// may be nil.
|
||||
func NewClient(baseURL string, cfg *compass.Config) (*Client, error) {
|
||||
baseURL = strings.TrimRight(strings.TrimSpace(baseURL), "/")
|
||||
if baseURL == "" {
|
||||
return nil, ErrRegistryDisabled
|
||||
}
|
||||
u, err := url.Parse(baseURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("kiosk: registry URL %q: %w", baseURL, err)
|
||||
}
|
||||
switch {
|
||||
case u.Scheme != "http" && u.Scheme != "https":
|
||||
return nil, fmt.Errorf("kiosk: registry URL %q must start with https:// or http://", baseURL)
|
||||
case u.Host == "":
|
||||
return nil, fmt.Errorf("kiosk: registry URL %q has no host", baseURL)
|
||||
case u.User != nil:
|
||||
return nil, fmt.Errorf("kiosk: registry URL must not carry user info")
|
||||
case u.RawQuery != "" || u.Fragment != "" || u.ForceQuery:
|
||||
return nil, fmt.Errorf("kiosk: registry URL %q must not have a query or fragment", baseURL)
|
||||
}
|
||||
hc, err := fetchguard.NewClient(fetchguard.Policy{
|
||||
Mode: fetchguard.TrustedMode,
|
||||
MaxBytes: registryMaxBytes,
|
||||
Timeout: registryTimeout,
|
||||
}, cfg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("kiosk: registry client: %w", err)
|
||||
}
|
||||
return &Client{base: baseURL, http: hc}, nil
|
||||
}
|
||||
|
||||
// BaseURL returns the registry origin the client calls.
|
||||
func (c *Client) BaseURL() string {
|
||||
return c.base
|
||||
}
|
||||
|
||||
// List returns the registry's plugins matching the search q and the
|
||||
// category (both optional).
|
||||
func (c *Client) List(ctx context.Context, q, category string) (ListResponse, error) {
|
||||
query := url.Values{}
|
||||
if q != "" {
|
||||
query.Set("q", q)
|
||||
}
|
||||
if category != "" {
|
||||
query.Set("category", category)
|
||||
}
|
||||
path := "/plugins"
|
||||
if len(query) > 0 {
|
||||
path += "?" + query.Encode()
|
||||
}
|
||||
var resp ListResponse
|
||||
err := c.get(ctx, path, "", &resp)
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// Plugin returns one plugin with its README and the requirements of its
|
||||
// latest release. An unknown ID gives an error wrapping ErrNotFound.
|
||||
func (c *Client) Plugin(ctx context.Context, id string) (RegistryPluginDetail, error) {
|
||||
var resp DetailResponse
|
||||
err := c.get(ctx, "/plugins/"+url.PathEscape(id), id, &resp)
|
||||
return resp.Data, err
|
||||
}
|
||||
|
||||
// Versions returns every release of a plugin, newest first. An unknown ID
|
||||
// gives an error wrapping ErrNotFound.
|
||||
func (c *Client) Versions(ctx context.Context, id string) ([]RegistryVersion, error) {
|
||||
var resp VersionsResponse
|
||||
err := c.get(ctx, "/plugins/"+url.PathEscape(id)+"/versions", id, &resp)
|
||||
return resp.Data, err
|
||||
}
|
||||
|
||||
func (c *Client) get(ctx context.Context, path, id string, out any) error {
|
||||
header := http.Header{}
|
||||
header.Set("Accept", "application/json")
|
||||
res, err := c.http.Get(ctx, c.base+RegistryAPIPath+path, header)
|
||||
if err != nil {
|
||||
return fmt.Errorf("registry %s: %w", c.base, err)
|
||||
}
|
||||
switch {
|
||||
case res.StatusCode == http.StatusNotFound:
|
||||
if id != "" {
|
||||
return fmt.Errorf("%s is %w at %s", id, ErrNotFound, c.base)
|
||||
}
|
||||
return fmt.Errorf("registry %s: %w", c.base, ErrNotFound)
|
||||
case res.StatusCode >= 300 && res.StatusCode < 400:
|
||||
return fmt.Errorf("registry %s answered with a redirect to %q; set plugins.registry_url to the final origin", c.base, res.Header.Get("Location"))
|
||||
case res.StatusCode < 200 || res.StatusCode >= 300:
|
||||
var e ErrorResponse
|
||||
if json.Unmarshal(res.Body, &e) == nil && e.Error.Code != "" {
|
||||
return fmt.Errorf("registry %s: HTTP %d %s: %s", c.base, res.StatusCode, e.Error.Code, e.Error.Message)
|
||||
}
|
||||
return fmt.Errorf("registry %s: HTTP %d", c.base, res.StatusCode)
|
||||
}
|
||||
if err := json.Unmarshal(res.Body, out); err != nil {
|
||||
return fmt.Errorf("registry %s: decode %s: %w", c.base, path, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user