feat(19.3-03): summer.yaml build.tags and the summercms_s3 tag that links the S3 uploads driver
- Manifest.Tags from build.tags (^[a-z0-9_]+$, duplicates dropped), round-tripped by SaveManifest - summer build passes -tags=<list> through goBuildArgs and prints the build tags line - attach: s3blob imported only by s3_on.go (//go:build summercms_s3); an untagged binary refuses an s3:// bucket_url naming the tag, build.tags and -tags - go mod tidy adds the AWS SDK v2 modules as indirect requirements - storage, modules, scaffolding, configuration and scaling docs plus the lagoon README
This commit is contained in:
@@ -69,7 +69,7 @@ There is no `go.sum` in the application root: builds go through the workspace, a
|
||||
|
||||
## The summer.yaml manifest
|
||||
|
||||
The manifest in the application root names the application module, the binary `summer build` writes to `bin/`, and the plugins in activation order:
|
||||
The manifest in the application root names the application module, the binary `summer build` writes to `bin/`, the plugins in activation order and, optionally, Go build tags:
|
||||
|
||||
```yaml
|
||||
module: git.golem15.com/acme/acme
|
||||
@@ -79,10 +79,15 @@ plugins:
|
||||
module: git.golem15.com/acme/acme/plugins/user
|
||||
- id: acme.blog
|
||||
module: git.golem15.com/acme/acme/plugins/blog
|
||||
build:
|
||||
tags:
|
||||
- summercms_s3
|
||||
```
|
||||
|
||||
`summer build` turns this list into `plugins.gen.go`. The order is the manifest order, adjusted so that each plugin comes after the plugins it requires. Plugins with no dependency between them keep the order you wrote.
|
||||
|
||||
`build.tags` lists Go build tags that `summer build` (and `summer dev`) passes to `go build` as `-tags=<tag>,<tag>`, printing them as `build tags: …`. Each tag must consist of lower-case letters, digits and underscores; a duplicate is ignored. Framework features that pull in large dependencies sit behind a tag, such as `summercms_s3` for S3 uploads buckets (see [Storage](../services/storage.md#s3-and-s3-compatible-buckets)). `summer plugin:add` keeps the list when it rewrites the manifest. A script that runs its own `go build` must pass the same tags.
|
||||
|
||||
## Local plugins in a workspace
|
||||
|
||||
A plugin you develop inside the application lives in `plugins/<name>` as its own module. `summer make:plugin acme.blog` creates it there, and `summer plugin:add plugins/blog` registers it:
|
||||
|
||||
@@ -68,7 +68,7 @@ Some costs come over with the port unchanged, and a few framework behaviours nee
|
||||
The application is one stateless binary. It starts in milliseconds, ships in a small container image and needs no OPcache warm-up after a deploy.
|
||||
|
||||
- Vertical scaling is more cores. The Go runtime uses them all, so there is no worker count to tune.
|
||||
- Horizontal scaling is N replicas behind a load balancer that share Postgres. Uploads must be shared too: point `storage.uploads.bucket_url` at a `file://` directory on storage that every replica mounts (see [Storage](../services/storage.md#bucket-urls)). Apply the blacklist and rate-limit notes above before you add the second replica.
|
||||
- Horizontal scaling is N replicas behind a load balancer that share Postgres. Uploads must be shared too: point `storage.uploads.bucket_url` at a `file://` directory on storage that every replica mounts, or at an S3 or S3-compatible bucket (`s3://`, in a binary built with the `summercms_s3` tag; see [Storage](../services/storage.md#s3-and-s3-compatible-buckets)). Apply the blacklist and rate-limit notes above before you add the second replica.
|
||||
- Centrifugo scales on its own. [lighthouse](../../modules/lighthouse/README.md) only publishes to it and issues connection tokens, so adding application replicas does not change the realtime server; see [The Centrifugo driver](../services/realtime.md#the-centrifugo-driver).
|
||||
|
||||
A rollout runs the migrations once, then starts each replica behind the reverse proxy:
|
||||
|
||||
@@ -12,8 +12,8 @@ The `summer` tool creates and builds applications and generates the files a plug
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `summer build` | Reads `summer.yaml`, generates `plugins.gen.go` and `main.go`, and builds the binary into `bin/`. Run it from the application directory or any directory below it. |
|
||||
| `summer dev` | Builds the application, starts the binary, and rebuilds and restarts it whenever a Go or YAML source, `go.mod`, `go.work`, `.env` or `summer.yaml` changes. |
|
||||
| `summer build` | Reads `summer.yaml`, generates `plugins.gen.go` and `main.go`, and builds the binary into `bin/`, passing the manifest's `build.tags` to `go build` as `-tags`. Run it from the application directory or any directory below it. |
|
||||
| `summer dev` | Builds the application as `summer build` does (build tags included), starts the binary, and rebuilds and restarts it whenever a Go or YAML source, `go.mod`, `go.work`, `.env` or `summer.yaml` changes. |
|
||||
|
||||
```sh
|
||||
summer build
|
||||
@@ -44,7 +44,7 @@ The application gets these files:
|
||||
|------|----------|
|
||||
| `go.mod` | Module `git.golem15.com/golem15/sm-<name>-app`, requiring the framework through a relative `replace`. |
|
||||
| `go.work` | The workspace with the application and its plugin modules, as relative paths. It is committed. |
|
||||
| `summer.yaml` | The manifest: module path, binary name and plugin list. |
|
||||
| `summer.yaml` | The manifest: module path, binary name, plugin list and optional `build.tags` (for example `summercms_s3` for an S3 uploads bucket). |
|
||||
| `config/` | `app.yaml`, `database.yaml`, `http.yaml` and `storage.yaml`, enough for `serve` and `route:list`. |
|
||||
| `.env`, `.env.example` | The database DSN, a freshly generated application key and a random admin token secret (`SUMMER_ADMIN__JWT__SECRET`). `.env` is readable by its owner only and never committed; `.env.example` lists the variables without the key and the secret. |
|
||||
| `compose.yaml` | PostgreSQL 16 on `127.0.0.1:54320`, with credentials matching `.env`. |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Storage
|
||||
description: Configure the uploads bucket that serve opens, choose file or memory bucket URLs, serve stored files, and size upload routes.
|
||||
description: Configure the uploads bucket that serve opens, choose file, memory or S3 bucket URLs, serve stored files, and size upload routes.
|
||||
section: services
|
||||
order: 90
|
||||
---
|
||||
@@ -14,6 +14,7 @@ WinterCMS stores uploads on a Laravel filesystem disk. SummerCMS stores them in
|
||||
|-----|--------|
|
||||
| `file:///var/lib/acme/uploads` | In a directory on the server. |
|
||||
| `mem://` | In memory, for tests. Everything is lost when the process exits. |
|
||||
| `s3://acme-uploads?region=eu-central-1` | In an S3 or S3-compatible bucket. The binary must be built with the `summercms_s3` tag; see [S3 and S3-compatible buckets](#s3-and-s3-compatible-buckets). |
|
||||
|
||||
The keys go in `config/storage.yaml`:
|
||||
|
||||
@@ -39,6 +40,45 @@ An original then has the URL `/storage/app/uploads/public/<partition>/<disk_name
|
||||
|
||||
Application code gets the bucket with `app.Lookup[*blob.Bucket]()` and reads and writes it through the `gocloud.dev/blob` API. Model attachments, thumbnails and deleting files after commit are covered in [Attachments](../database/attachments.md).
|
||||
|
||||
## S3 and S3-compatible buckets
|
||||
|
||||
The S3 driver (`gocloud.dev/blob/s3blob`) is not linked into every binary, since it brings in the AWS SDK. A binary links it when it is built with the Go build tag `summercms_s3` (`attach.S3BuildTag`). The application never imports the driver itself: list the tag under `build.tags` in `summer.yaml` and `summer build` passes it to `go build`:
|
||||
|
||||
```yaml
|
||||
module: git.golem15.com/acme/acme
|
||||
binary: acme
|
||||
plugins:
|
||||
- id: acme.blog
|
||||
module: git.golem15.com/acme/acme/plugins/blog
|
||||
build:
|
||||
tags:
|
||||
- summercms_s3
|
||||
```
|
||||
|
||||
`summer build` prints `build tags: summercms_s3` before it builds, and `summer dev` builds the same way. A build script that runs its own `go build` after `summer build`, for example to cross-compile a release, must repeat the tag: `go build -tags summercms_s3 -o bin/acme .`, or `GOFLAGS=-tags=summercms_s3` in its environment. A binary built without the tag refuses an `s3://` bucket URL at start-up:
|
||||
|
||||
```text
|
||||
attach: storage.uploads.bucket_url uses s3:// but this binary was built without the summercms_s3 build tag; list summercms_s3 under build.tags in summer.yaml and rebuild with summer build (a raw go build needs -tags summercms_s3, or GOFLAGS=-tags=summercms_s3)
|
||||
```
|
||||
|
||||
The bucket is configured only through `bucket_url` and its query parameters, as gocloud.dev defines them; there are no separate `s3.*` keys. For AWS:
|
||||
|
||||
```yaml
|
||||
uploads:
|
||||
bucket_url: s3://acme-uploads?region=eu-central-1
|
||||
```
|
||||
|
||||
For an S3-compatible service (MinIO, Cloudflare R2, Hetzner Object Storage and others), add the service's `endpoint`, path-style addressing and, for services that do not support the newer AWS checksums, the two checksum parameters:
|
||||
|
||||
```yaml
|
||||
uploads:
|
||||
bucket_url: s3://acme-uploads?region=auto&endpoint=https://storage.example.com&use_path_style=true&request_checksum_calculation=when_required&response_checksum_validation=when_required
|
||||
```
|
||||
|
||||
Credentials come from the standard AWS environment: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`, a profile (`AWS_PROFILE`, or `profile=` in the URL) or an instance role. Never put keys in `bucket_url` or in a config file. The bucket opens at start-up without a request to the service, so wrong credentials or a missing bucket show up on the first upload or read.
|
||||
|
||||
The [Serving files](#serving-files) route reads through the bucket, so it serves an S3 bucket exactly as it serves a directory.
|
||||
|
||||
## Serving files
|
||||
|
||||
The framework mounts no file route unless you ask for one. Set `storage.uploads.serve` to `true` (environment `SUMMER_STORAGE__UPLOADS__SERVE=true`, default `false`) and the router registers `GET <public_path_prefix>/{path...}` on `attach.StaticHandlerPublic`:
|
||||
|
||||
@@ -57,7 +57,7 @@ These are the keys most applications set. Each module's reference page lists all
|
||||
| `cache.driver`, `cache.prefix`, `cache.memory.max_entries` | The cache driver (`memory`, the default, or `redis`), the key prefix (default from `app.name`) and the memory driver's size bound (default 10000). | [cooler](../../modules/cooler/README.md) |
|
||||
| `redis.url` | Redis URL (`redis://`, `rediss://` or `unix://`), required when a Redis cache or rate-limit store is selected. Keep it in `SUMMER_REDIS__URL` when it carries a password. | [cooler](../../modules/cooler/README.md) |
|
||||
| `http.access_log` | Logs one record per request with method, matched route pattern, status and duration. Defaults to `false`. | [surf](../../modules/surf/README.md) |
|
||||
| `storage.uploads.bucket_url` | Uploads bucket, `file://` or `mem://`. Required by `serve`. | [lagoon](../../modules/lagoon/README.md) |
|
||||
| `storage.uploads.bucket_url` | Uploads bucket, `file://`, `mem://` or `s3://` (the last only in a binary built with the `summercms_s3` tag from `build.tags` in `summer.yaml`; credentials from the AWS environment). Required by `serve`. | [lagoon](../../modules/lagoon/README.md) |
|
||||
| `storage.uploads.serve` | Mounts `attach.StaticHandlerPublic` under `storage.uploads.public_path_prefix` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Defaults to `false`. | [surf](../../modules/surf/README.md) |
|
||||
| `admin.jwt.secret` | Secret for admin tokens. Required as soon as a plugin registers an admin controller. | [cabana](../../modules/cabana/README.md) |
|
||||
| `backend.uri` | Path the admin is served under. Defaults to `/backend`. | [cabana](../../modules/cabana/README.md) |
|
||||
|
||||
Reference in New Issue
Block a user