feat(surf): serve startup summary, shutdown report and http request log
- serve prints a startup summary on a terminal (version, environment, URL, admin, database host and name, pending migrations, plugins, drivers) and logs it as a 'startup summary' record plus 'listening on <addr>' - on SIGINT/SIGTERM serve drains, prints and logs a shutdown report and returns nil (exit 0), or the drain errors (exit 1) - the request log is on by default under diary component http with request_id; the route pattern, never the path, is logged - lagoon.PendingMigrations, conga.Worker.Stats, diary.HideOnTTY, bonfire.IsTerminal and bonfire.Details back the summary and report
This commit is contained in:
Jakub Zych
@@ -62,7 +62,7 @@ summer migrate
|
||||
summer serve
|
||||
```
|
||||
|
||||
The next steps also explain the admin. It opens at `/backend` once a plugin registers an admin controller; until then `summer serve` prints `admin: off` with these steps:
|
||||
The next steps also explain the admin. It opens at `/backend` once a plugin registers an admin controller; until then the Admin line of `summer serve`'s startup summary reads `off`, followed by these steps:
|
||||
|
||||
```sh
|
||||
summer make:admin-controller golem15.acme Posts
|
||||
|
||||
@@ -55,7 +55,7 @@ Copy the printed value into `SUMMER_APP__KEY`. When you rotate the key, keep the
|
||||
|
||||
| Command | Flags | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, builds the router, binds the address, starts the in-process job worker and serves HTTP until SIGINT or SIGTERM, then shuts down within 10 seconds. After binding it prints the listen address with a browser URL (an unspecified host shows as 127.0.0.1) and an admin status line: the admin URL, or `admin: off` with the steps to add an admin controller. |
|
||||
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, builds the router, binds the address, starts the in-process job worker and serves HTTP until SIGINT or SIGTERM. Once it listens it prints a startup summary; on a signal it drains requests and the worker within 10 seconds and prints a shutdown report. |
|
||||
| `route:list` | none | Builds the router the way `serve` does, without opening the database or listening, and prints every route with its method, pattern, plugin, middleware and raw flag, followed by the admin status line. |
|
||||
|
||||
```sh
|
||||
@@ -67,6 +67,44 @@ The default `--addr` listens on every interface. Pass a loopback address during
|
||||
|
||||
From the application directory, `summer serve --addr 127.0.0.1:8123` passes the flag through to the built binary.
|
||||
|
||||
### Startup summary
|
||||
|
||||
On a terminal `serve` prints a block once the port is bound:
|
||||
|
||||
```text
|
||||
SummerCMS acme v1.4.0 (framework v0.4.0)
|
||||
Environment production
|
||||
URL http://127.0.0.1:8080/
|
||||
Admin http://127.0.0.1:8080/backend
|
||||
Database db.internal:5432/acme
|
||||
Migrations up to date
|
||||
Plugins 3 (acme.user, acme.blog, acme.shop)
|
||||
Queue default, mail, scheduled (worker in serve)
|
||||
Scheduler in serve
|
||||
Mail smtp
|
||||
Cache redis
|
||||
Realtime null
|
||||
Search null
|
||||
```
|
||||
|
||||
The database line shows the host, port and database name, never the user or password. Pending migrations read `3 pending — run bin/acme migrate`. The admin line shows the admin URL, or `off` with the steps to add an admin controller; a warning naming `summer admin:create` follows when the `backend_users` table is empty. `-v` adds one line per plugin with the plugins it requires and its pending migrations, and lets every component's info logs through; `-q` prints no summary, only warnings and errors.
|
||||
|
||||
Off a terminal (systemd, Docker, a redirected log) nothing is printed on stdout: the same facts are logged as one `startup summary` record and one record whose message is exactly `listening on <addr>`, both from the `serve` component, so journald and the daily files keep them and scripts can wait for the second one. On a terminal the two records still reach the daily files, but the console does not repeat them.
|
||||
|
||||
### Shutting down
|
||||
|
||||
Ctrl+C or SIGTERM drains the requests in flight for up to 10 seconds, stops the in-process worker and prints a report (on a terminal) and a `shutdown report` record:
|
||||
|
||||
```text
|
||||
Shutdown report
|
||||
Requests 1520 served, 2 drained, 0 aborted
|
||||
Worker 84 done, 1 failed, 0 handed back to the queue
|
||||
Uptime 3h12m40s
|
||||
Errors none
|
||||
```
|
||||
|
||||
Jobs still running at the deadline are cancelled; River records the cancelled attempt and retries the job later, which the report counts as handed back to the queue. `serve` exits 0 after a clean stop and 1 when draining fails (requests still running at the deadline, or the worker failing to stop).
|
||||
|
||||
## Admin accounts
|
||||
|
||||
| Command | Arguments and flags | Purpose |
|
||||
@@ -88,7 +126,7 @@ TOKEN=$(./bin/acme admin:token:create admin@example.com --name agent --scope blo
|
||||
|
||||
From the application directory, `summer admin:create --email admin@example.com --superuser` runs the same command on the built binary, prompt included. The four `admin:token:*` commands have matching `summer` delegates.
|
||||
|
||||
When the admin is mounted and the `backend_users` table is empty, `serve` prints a warning at start-up naming `summer admin:create`.
|
||||
When the admin is mounted and the `backend_users` table is empty, `serve` prints a warning after its startup summary naming `summer admin:create`.
|
||||
|
||||
## Queues and the scheduler
|
||||
|
||||
|
||||
@@ -146,14 +146,7 @@ Redaction is a safety net for mistakes, not a way to log secrets on purpose. Log
|
||||
|
||||
## Request log
|
||||
|
||||
The router can log every request. It is off by default; turn it on with the `http.access_log` key (environment `SUMMER_HTTP__ACCESS_LOG=true`):
|
||||
|
||||
```yaml
|
||||
# config/http.yaml
|
||||
access_log: true
|
||||
```
|
||||
|
||||
Each request then writes one Info record with the message `http request` once it has been served:
|
||||
The router logs every request. Each request writes one Info record with the message `http request` from the component `http` once it has been served:
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
@@ -161,9 +154,25 @@ Each request then writes one Info record with the message `http request` once it
|
||||
| `pattern` | The matched route, method and path pattern, for example `GET /posts/{id}`. Empty when no route matched. |
|
||||
| `status` | The response status; a handler that writes without setting one counts as 200. A 5xx is logged as 499 (client closed request) only when the client had already gone away when the status was written and the handler did not panic: the error came from the cancellation, and no client received it. A 5xx written before the client went away keeps its status, and a recovered panic is always 500, so a search for 5xx records finds every real server error. |
|
||||
| `duration_ms` | Time spent serving the request, in whole milliseconds. |
|
||||
| `request_id` | The `X-Request-Id` request header, trimmed and cut to 128 bytes; absent when the header is empty. Set it in your reverse proxy to follow one request across the proxy's and the application's logs. |
|
||||
| `client_gone` | `true` when the client went away before the request was served (its context ended with `context.Canceled`); absent otherwise. |
|
||||
|
||||
The record never carries the URL path, the query string, request or response headers or the client address, because tokens often travel in paths and query strings. That holds for a request that reached no route too: a token route requested with another method gets a 405 from the router, and its trailing-slash variant a 404, before any route handler runs, so the path would carry the token. It goes to the published `*slog.Logger` or `slog.Default`, so the redacting handler still applies. Grep for the empty `pattern` to find requests that reached no route; the client's own log (a browser, a proxy) shows which path it asked for.
|
||||
The record carries the route pattern, never the URL path, the query string, request or response headers or the client address, because tokens travel in paths and query strings: share and invitation links carry them in the path. That holds for a request that reached no route too: a token route requested with another method gets a 405 from the router, and its trailing-slash variant a 404, before any route handler runs, so the path would carry the token. Grep for the empty `pattern` to find requests that reached no route; the client's own log (a browser, a proxy) shows which path it asked for.
|
||||
|
||||
The log is on by default, at Info, so the console shows it and the daily files keep what their level lets through. Silence it with the component level, or turn it off with `http.access_log` (environment `SUMMER_HTTP__ACCESS_LOG=false`):
|
||||
|
||||
```yaml
|
||||
# config/log.yaml
|
||||
components:
|
||||
http: warn
|
||||
```
|
||||
|
||||
```yaml
|
||||
# config/http.yaml
|
||||
access_log: false
|
||||
```
|
||||
|
||||
It goes to the published `*slog.Logger` when a plugin publishes one, else to the `http` component logger, so the redacting handler applies on every channel.
|
||||
|
||||
## Panics
|
||||
|
||||
|
||||
@@ -68,7 +68,7 @@ curl http://127.0.0.1:8080/
|
||||
|
||||
It returns `{"message":"Hello from SummerCMS","plugin":"golem15.demo"}`. Stop the database with `docker compose down`, or `docker compose down -v` to delete its data as well.
|
||||
|
||||
`summer serve` prints the URL to open and the admin status. The admin lives at `http://127.0.0.1:8080/backend` once a plugin registers an admin controller; until then the status line reads `admin: off` and names the steps. When the admin is there, create the first administrator:
|
||||
`summer serve` prints a startup summary with the URL to open and the admin status. The admin lives at `http://127.0.0.1:8080/backend` once a plugin registers an admin controller; until then the summary's Admin line reads `off` and the steps follow it. When the admin is there, create the first administrator:
|
||||
|
||||
```sh
|
||||
summer admin:create --email you@example.com --superuser
|
||||
|
||||
@@ -157,6 +157,8 @@ if catalog.Has("blog:import") {
|
||||
| `bonfire.GlobalsOf` | The `bonfire.Globals` of the command an Output belongs to (zero for other Output implementations). |
|
||||
| `bonfire.VerbosityOf` | The `-v` level of the command an Output belongs to. |
|
||||
| `bonfire.IsQuiet` | Whether the command runs with `-q`. |
|
||||
| `bonfire.IsTerminal` | Whether the Output writes to a terminal; long-running commands print a summary block for a person there and log records otherwise. |
|
||||
| `bonfire.Details`, `bonfire.Detail` | Prints a title and aligned label/value lines in the palette (a `Warn` value in the warning color); nothing under `-q`. Used for startup summaries and shutdown reports. |
|
||||
| `bonfire.Interactive` | Whether the Output can ask a person: stdin and stdout are terminals and neither `-n` nor `-q` is set. |
|
||||
| `bonfire.ConfirmProceed` | Asks before a destructive step on an interactive terminal; returns true without asking otherwise. |
|
||||
| `bonfire.ErrAborted` | Returned by a prompt the person cancelled. |
|
||||
|
||||
@@ -3,6 +3,7 @@ package bonfire
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
tea "charm.land/bubbletea/v2"
|
||||
)
|
||||
@@ -13,12 +14,15 @@ import (
|
||||
type consoleState interface {
|
||||
consoleGlobals() Globals
|
||||
consoleInteractive() bool
|
||||
consoleTerminal() bool
|
||||
}
|
||||
|
||||
func (c *console) consoleGlobals() Globals { return c.globals }
|
||||
|
||||
func (c *console) consoleInteractive() bool { return c.canPrompt() }
|
||||
|
||||
func (c *console) consoleTerminal() bool { return c.tty }
|
||||
|
||||
// GlobalsOf returns the global options of the command out belongs to, or
|
||||
// the zero Globals when out is not a bonfire console.
|
||||
func GlobalsOf(out Output) Globals {
|
||||
@@ -48,6 +52,66 @@ func Interactive(out Output) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// IsTerminal reports whether out writes to a terminal (its stdout is one).
|
||||
// It is false for other Output implementations and inside bonfire.Call.
|
||||
// Long-running commands use it to choose between a styled summary block for
|
||||
// a person and log records for journald.
|
||||
func IsTerminal(out Output) bool {
|
||||
if s, ok := out.(consoleState); ok {
|
||||
return s.consoleTerminal()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Detail is one line of a Details block: a label and its value. Warn
|
||||
// prints the value in the warning color.
|
||||
type Detail struct {
|
||||
Label string
|
||||
Value string
|
||||
Warn bool
|
||||
}
|
||||
|
||||
// Details prints title (when not empty) and rows as aligned label/value
|
||||
// lines in the console palette: muted labels padded to the longest one,
|
||||
// values in the default color or, with Warn, in the warning color. It
|
||||
// prints nothing under -q. serve, queue:work and summer dev use it for
|
||||
// their startup summaries and shutdown reports.
|
||||
func Details(out Output, title string, rows []Detail) {
|
||||
if IsQuiet(out) {
|
||||
return
|
||||
}
|
||||
width := 0
|
||||
for _, r := range rows {
|
||||
width = max(width, len([]rune(r.Label)))
|
||||
}
|
||||
c, styled := out.(*console)
|
||||
var b strings.Builder
|
||||
if title != "" {
|
||||
if styled {
|
||||
b.WriteString(c.palette.Header.Render(title))
|
||||
} else {
|
||||
b.WriteString(title)
|
||||
}
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
for _, r := range rows {
|
||||
label := r.Label + strings.Repeat(" ", width-len([]rune(r.Label)))
|
||||
value := r.Value
|
||||
if styled {
|
||||
label = c.palette.Muted.Render(label)
|
||||
if r.Warn {
|
||||
value = c.palette.Warning.Render(value)
|
||||
}
|
||||
}
|
||||
b.WriteString(" " + label + " " + value + "\n")
|
||||
}
|
||||
if styled {
|
||||
c.emit(c.out, b.String())
|
||||
return
|
||||
}
|
||||
out.Printf("%s", b.String())
|
||||
}
|
||||
|
||||
// ConfirmProceed asks question before a destructive step, defaulting to
|
||||
// no, only when out is Interactive. Off a terminal, with -n or -q and in
|
||||
// bonfire.Call it returns true without asking, so deploys over SSH and
|
||||
|
||||
@@ -34,7 +34,7 @@ Use cases:
|
||||
- The WinterCMS job manager operations with the same semantics: `conga.Manager.StartJob`, `conga.Manager.UpdateJobState`, `conga.Manager.UpdateMetadata`, `conga.Manager.CompleteJob`, `conga.Manager.FailJob`, `conga.Manager.CheckIfCanceled` and `conga.Manager.GetMetadata`. Updates are raw column writes, so `updated_at` changes only on dispatch and `conga.Manager.StartJob`.
|
||||
- Cancellation in two parts: `conga.Manager.CancelJob` is the outside cancel (sets `is_canceled` and `conga.StatusStopped`, then cancels the River job, so a queued job never starts and a running job's context is cancelled); `conga.Manager.StopJob` is what a job calls on its own row after `conga.Manager.CheckIfCanceled` reports true (status only, the WinterCMS `cancelJob`).
|
||||
- Outcome rules in the worker: an error on an attempt before the last leaves the row in progress so River can retry; the final failed attempt, or a recovered panic on it, sets `conga.StatusError` with the error text under the metadata key `error`; an error on a row that was stopped or cancelled cancels the River job instead. A job that returns nil without completing its row leaves it as it is. Skipped work is recorded as complete with `{"skipped": true}` metadata.
|
||||
- Workers: `conga.StartWorker` registers every plugin job and starts one River client; `conga.WorkerOptions.Queues` limits it to some queues, and an unknown queue is `conga.ErrUnknownQueue` listing the known ones. `conga.Worker.Stop` stops it gracefully and cancels running jobs when its context ends. `conga.StartServeWorker` is the variant the `serve` command uses: it starts nothing when `queue.work_in_serve` is false. An app without jobs still gets a worker that starts and idles.
|
||||
- Workers: `conga.StartWorker` registers every plugin job and starts one River client; `conga.WorkerOptions.Queues` limits it to some queues, and an unknown queue is `conga.ErrUnknownQueue` listing the known ones. `conga.Worker.Stop` stops it gracefully and cancels running jobs when its context ends; `conga.Worker.Stats` counts attempts done, failed, in flight and handed back to the queue by that cancellation. `conga.StartServeWorker` is the variant the `serve` command uses: it starts nothing when `queue.work_in_serve` is false. An app without jobs still gets a worker that starts and idles.
|
||||
- Scheduled commands: every worker carries one River periodic job per `pact.HasSchedule` entry, in plugin activation order then declaration order, with the id `<plugin id>[<index>]:<command>`. Only the elected leader enqueues. Each run is a `conga.ScheduledCommandArgs` job on `conga.QueueScheduled` with one attempt (an interrupted run is not retried; the next period runs normally) and unique by args within its cadence period, so a leader failover cannot double-enqueue a period. The worker runs a job only when its entry exists in the compiled schedule and its command and arguments match that entry exactly, so a forged `river_job` row cannot run an arbitrary command. An unregistered command, or an app with no published `bonfire.Catalog`, is logged at Warn (`schedule: command not registered; skipping`) and skipped without failing the worker or other entries. Command output is logged line by line at Info with a `command` attribute; failures are logged with the duration.
|
||||
- Framework schedule entries: before the plugin entries, every worker carries the framework's own entries from `lagoon.FrameworkSchedule`, listed under the plugin id `lagoon.FrameworkScheduleID`. Today that is the daily `deferred:purge` at `database.deferred_bindings.purge_at` (default `03:00`), with the id `summercms.lagoon[0]:deferred:purge`; it runs, appears in `schedule:run --once` and is protected by the compiled entry table exactly like a plugin entry. An empty `purge_at` removes it, and a malformed one fails the worker start.
|
||||
- Wall-clock schedules: `conga.Daily` fires at the next `hh:mm` in its location and `conga.Every` at the next multiple of its interval since local midnight, so a restart never delays a daily run by up to a day the way `river.PeriodicInterval(24h)` would. On a DST day `conga.Daily` keeps the wall-clock time. A schedule entry with an empty command, a zero cadence, a daily time out of range, or an interval under one second or not dividing 24h fails the worker start with an error naming the plugin id and entry index.
|
||||
@@ -183,7 +183,8 @@ defer w.Stop(context.Background())
|
||||
| `conga.StartServeWorker` | The worker of the `serve` command; nil when `queue.work_in_serve` is false. |
|
||||
| `conga.WorkerOptions` | Selects the queues a worker runs. |
|
||||
| `conga.RuntimeCommands` | Returns the `queue:work`, `schedule:run` and `queue:clear` commands. |
|
||||
| `conga.Worker` | A running worker; `conga.Worker.Stop` stops it and `conga.Worker.Queues` lists its queues. |
|
||||
| `conga.Worker` | A running worker; `conga.Worker.Stop` stops it, `conga.Worker.Queues` lists its queues and `conga.Worker.Stats` returns its attempt counters. |
|
||||
| `conga.WorkerStats` | Attempt counters since the worker started: `Done`, `Failed`, `InFlight` and `HandedBack` (attempts cancelled when `Stop` reached its deadline; River records them as errored attempts and retries them). The shutdown reports of `serve` and `queue:work` print them. |
|
||||
| `conga.Daily` | `river.PeriodicSchedule` firing at `Hour:Minute` every day in `Loc` (UTC when nil). |
|
||||
| `conga.Every` | `river.PeriodicSchedule` firing at every multiple of `Interval` since local midnight in `Loc` (UTC when nil). |
|
||||
| `conga.ScheduledCommandArgs` | Args of one scheduled run: compiled `Entry` id, `Command` and `Args`; kind `summer.scheduled_command`. |
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
@@ -55,6 +56,9 @@ type Manager struct {
|
||||
// compiled entry table of the running worker, keyed by entry id.
|
||||
scheduledJob pact.Job
|
||||
schedule map[string]pact.ScheduledCommand
|
||||
|
||||
// counters are the attempt counters of the running worker.
|
||||
counters atomic.Pointer[workerCounters]
|
||||
}
|
||||
|
||||
// From returns the app's Manager, publishing a new one on first use.
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"fmt"
|
||||
"runtime/debug"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
@@ -38,6 +39,44 @@ type Worker struct {
|
||||
client *river.Client[*sql.Tx]
|
||||
listener *pgxpool.Pool
|
||||
queues []string
|
||||
counters *workerCounters
|
||||
}
|
||||
|
||||
// WorkerStats counts the job attempts a worker ran since it started.
|
||||
type WorkerStats struct {
|
||||
// Done is the number of attempts that returned no error.
|
||||
Done int64
|
||||
// Failed is the number of attempts that returned an error, other than
|
||||
// those cancelled by Stop (HandedBack).
|
||||
Failed int64
|
||||
// InFlight is the number of attempts running now.
|
||||
InFlight int64
|
||||
// HandedBack is the number of attempts cancelled by Stop after its
|
||||
// deadline (River's StopAndCancel). River records the cancellation as
|
||||
// an errored attempt, so the job is retried later, or fails for good
|
||||
// when it was its last attempt.
|
||||
HandedBack int64
|
||||
}
|
||||
|
||||
// workerCounters are the live counters behind WorkerStats.
|
||||
type workerCounters struct {
|
||||
done, failed, inFlight, handedBack atomic.Int64
|
||||
// stopping is set by Stop before running jobs are cancelled.
|
||||
stopping atomic.Bool
|
||||
}
|
||||
|
||||
// Stats returns the worker's attempt counters. A nil worker has none.
|
||||
func (w *Worker) Stats() WorkerStats {
|
||||
if w == nil || w.counters == nil {
|
||||
return WorkerStats{}
|
||||
}
|
||||
c := w.counters
|
||||
return WorkerStats{
|
||||
Done: c.done.Load(),
|
||||
Failed: c.failed.Load(),
|
||||
InFlight: c.inFlight.Load(),
|
||||
HandedBack: c.handedBack.Load(),
|
||||
}
|
||||
}
|
||||
|
||||
// Queues returns the sorted queue names the worker runs.
|
||||
@@ -152,9 +191,11 @@ func StartWorker(ctx context.Context, app *backpack.App, plugins []party.Plugin,
|
||||
closePool(listener)
|
||||
return nil, fmt.Errorf("conga: start worker: %w", err)
|
||||
}
|
||||
counters := &workerCounters{}
|
||||
m.worker = client
|
||||
m.schedule = table
|
||||
return &Worker{m: m, client: client, listener: listener, queues: sortedQueueNames(queues)}, nil
|
||||
m.counters.Store(counters)
|
||||
return &Worker{m: m, client: client, listener: listener, queues: sortedQueueNames(queues), counters: counters}, nil
|
||||
}
|
||||
|
||||
// StartServeWorker starts the in-process worker of the serve command on
|
||||
@@ -175,6 +216,9 @@ func (w *Worker) Stop(ctx context.Context) error {
|
||||
}
|
||||
err := w.client.Stop(ctx)
|
||||
if err != nil && ctx.Err() != nil {
|
||||
if w.counters != nil {
|
||||
w.counters.stopping.Store(true)
|
||||
}
|
||||
hardCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
|
||||
err = w.client.StopAndCancel(hardCtx)
|
||||
cancel()
|
||||
@@ -235,8 +279,8 @@ func selectQueues(known map[string]int, want []string) (map[string]int, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// runAttempt runs one River attempt of a job and applies the summer_jobs
|
||||
// rules: the row id goes into ctx; a panic becomes an error; an error on a
|
||||
// runAttempt runs one River attempt of a job, counts it in the running
|
||||
// worker's WorkerStats and applies the summer_jobs rules: the row id goes into ctx; a panic becomes an error; an error on a
|
||||
// STOPPED or canceled row cancels the River job; an error on the final
|
||||
// attempt sets StatusError with the error text under metadata "error". An
|
||||
// error on an earlier attempt leaves the row as it is so River can retry.
|
||||
@@ -245,7 +289,22 @@ func (m *Manager) runAttempt(ctx context.Context, row *rivertype.JobRow, fn func
|
||||
if hasRow {
|
||||
ctx = withJobID(ctx, id)
|
||||
}
|
||||
c := m.counters.Load()
|
||||
if c != nil {
|
||||
c.inFlight.Add(1)
|
||||
}
|
||||
err := m.call(ctx, row, fn)
|
||||
if c != nil {
|
||||
c.inFlight.Add(-1)
|
||||
switch {
|
||||
case err == nil:
|
||||
c.done.Add(1)
|
||||
case c.stopping.Load() && ctx.Err() != nil:
|
||||
c.handedBack.Add(1)
|
||||
default:
|
||||
c.failed.Add(1)
|
||||
}
|
||||
}
|
||||
if err == nil || !hasRow {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -89,6 +89,7 @@ Tests that set diary up restore the previous process logger with `t.Cleanup(diar
|
||||
| `diary.FromGlobals` | Returns the `diary.Options` (verbosity, quiet, color mode) of the console global options in a `bonfire.Globals`. |
|
||||
| `diary.ApplyGlobals` | Applies parsed `bonfire.Globals` to the running setup: verbosity and quiet, and the console color mode when it changed. |
|
||||
| `diary.Verbosity` | Returns the current verbosity. |
|
||||
| `diary.HideOnTTY` | Marks a context so a console channel writing to a terminal skips the records logged with it; files, stderr channels and a console that is not a terminal still write them. Used for facts a command has just printed as a block, such as `serve`'s startup summary. |
|
||||
| `diary.Reset` | Drops the setup, closes its files and restores the process default logger captured at the first setup. |
|
||||
| `diary.Close` | Closes the open daily files; later records open them again. |
|
||||
| `diary.Options` | Setup options: `Verbosity`, `Quiet`, `Color`, `Root`, `Stderr`, `Environ` and `Now`. |
|
||||
@@ -147,7 +148,7 @@ Environment variables use the `SUMMER_LOG__` prefix with a double underscore bet
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: `bonfire` (palette and color detection), `compass` (configuration), `sunscreen` (redaction).
|
||||
- Third-party: `charm.land/lipgloss/v2`, `github.com/charmbracelet/colorprofile` (decided in `.planning/notes/2026-10-21-console-charm-stack.md`).
|
||||
- Third-party: `charm.land/lipgloss/v2`, `github.com/charmbracelet/colorprofile` (decided in `.planning/notes/2026-10-21-console-charm-stack.md`), `golang.org/x/term` (whether a console channel writes to a terminal, for `diary.HideOnTTY`).
|
||||
- Standard library: `context`, `errors`, `fmt`, `io`, `io/fs`, `log`, `log/slog`, `maps`, `os`, `path/filepath`, `runtime`, `slices`, `strconv`, `strings`, `sync`, `sync/atomic`, `time`, `unicode`.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -110,6 +110,9 @@ func (b *channelBuilder) build(name string) (slog.Handler, error) {
|
||||
leaf = slog.NewTextHandler(w, hopts)
|
||||
}
|
||||
}
|
||||
if _, styled := leaf.(*consoleHandler); !styled && isTerminal(w) {
|
||||
leaf = ttyFilter{leaf}
|
||||
}
|
||||
b.out.consoles = append(b.out.consoles, consoleLevel{lv: lv, base: base})
|
||||
wrapped := sunscreen.Wrap(leaf)
|
||||
if b.warnTo == nil {
|
||||
|
||||
@@ -17,6 +17,7 @@ import (
|
||||
"charm.land/lipgloss/v2"
|
||||
"git.golem15.com/golem15/summercms/modules/bonfire"
|
||||
"github.com/charmbracelet/colorprofile"
|
||||
"golang.org/x/term"
|
||||
)
|
||||
|
||||
// componentWidth is the column width of the component prefix.
|
||||
@@ -113,7 +114,12 @@ func (h *consoleHandler) WithGroup(name string) slog.Handler {
|
||||
return c
|
||||
}
|
||||
|
||||
func (h *consoleHandler) Handle(_ context.Context, r slog.Record) error {
|
||||
func (h *consoleHandler) Handle(ctx context.Context, r slog.Record) error {
|
||||
// The styled console is read by a person; facts already shown as a
|
||||
// block (HideOnTTY) are not repeated as log lines.
|
||||
if hiddenOnTTY(ctx) {
|
||||
return nil
|
||||
}
|
||||
component := h.component
|
||||
prefix := groupPrefix(h.groups)
|
||||
var recAttrs []consoleAttr
|
||||
@@ -168,6 +174,37 @@ func (h *consoleHandler) Handle(_ context.Context, r slog.Record) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// ttyFilter drops records marked with HideOnTTY; it wraps a plain (JSON
|
||||
// or logfmt) console channel whose writer is a terminal.
|
||||
type ttyFilter struct {
|
||||
slog.Handler
|
||||
}
|
||||
|
||||
func (f ttyFilter) Enabled(ctx context.Context, l slog.Level) bool {
|
||||
return !hiddenOnTTY(ctx) && f.Handler.Enabled(ctx, l)
|
||||
}
|
||||
|
||||
func (f ttyFilter) Handle(ctx context.Context, r slog.Record) error {
|
||||
if hiddenOnTTY(ctx) {
|
||||
return nil
|
||||
}
|
||||
return f.Handler.Handle(ctx, r)
|
||||
}
|
||||
|
||||
func (f ttyFilter) WithAttrs(attrs []slog.Attr) slog.Handler {
|
||||
return ttyFilter{f.Handler.WithAttrs(attrs)}
|
||||
}
|
||||
|
||||
func (f ttyFilter) WithGroup(name string) slog.Handler {
|
||||
return ttyFilter{f.Handler.WithGroup(name)}
|
||||
}
|
||||
|
||||
// isTerminal reports whether w is a terminal.
|
||||
func isTerminal(w io.Writer) bool {
|
||||
f, ok := w.(interface{ Fd() uintptr })
|
||||
return ok && term.IsTerminal(int(f.Fd()))
|
||||
}
|
||||
|
||||
func levelLabel(l slog.Level, p bonfire.Palette) (string, lipgloss.Style) {
|
||||
switch {
|
||||
case l < slog.LevelInfo:
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
package diary
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"log"
|
||||
@@ -225,6 +226,30 @@ func ApplyGlobals(g bonfire.Globals) {
|
||||
SetVerbosity(g.Verbosity, g.Quiet)
|
||||
}
|
||||
|
||||
type hideOnTTYKey struct{}
|
||||
|
||||
// HideOnTTY marks ctx so that a console channel writing to a terminal skips
|
||||
// the records logged with it (Logger.InfoContext and the like); daily files,
|
||||
// stderr channels and a console that is not a terminal still write them. A
|
||||
// command that has just printed the same facts as a styled block (serve's
|
||||
// startup summary) logs them with it, so a person does not read them twice
|
||||
// while journald and the log files keep them.
|
||||
func HideOnTTY(ctx context.Context) context.Context {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
return context.WithValue(ctx, hideOnTTYKey{}, true)
|
||||
}
|
||||
|
||||
// hiddenOnTTY reports whether ctx carries the HideOnTTY mark.
|
||||
func hiddenOnTTY(ctx context.Context) bool {
|
||||
if ctx == nil {
|
||||
return false
|
||||
}
|
||||
v, _ := ctx.Value(hideOnTTYKey{}).(bool)
|
||||
return v
|
||||
}
|
||||
|
||||
// Verbosity returns the current verbosity (0 by default).
|
||||
func Verbosity() int {
|
||||
return int(verbosity.Load())
|
||||
|
||||
@@ -136,6 +136,7 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
|
||||
| `lagoon.Migrate` | Runs framework and plugin migrations in order. |
|
||||
| `lagoon.RollbackLast` | Rolls back the last migration of one plugin. |
|
||||
| `lagoon.Status` | Lists applied migration IDs per plugin as `lagoon.StatusRow` values. |
|
||||
| `lagoon.PendingMigrations` | Lists the migrations `lagoon.Migrate` would apply, as `lagoon.MigrationRef` values (`Plugin`, `ID`): the framework sets first, then each plugin in order, each in declaration order. A missing history table means the whole set is pending. Read-only; `serve` reports the count in its startup summary. |
|
||||
| `lagoon.HistoryTableName` | Returns the gormigrate history table for a plugin ID. |
|
||||
| `lagoon.BackendAdminMigrations` | Creates the backend user, role and admin token blacklist tables, the `backend_api_tokens` table for personal API tokens, seeds the system roles, adds a case-insensitive unique index on `backend_users.email` and adds WinterCMS's nullable jsonable `backend_users.metadata` column (TEXT; the admin subtitle lives under its `subtitle` key). The index migration refuses to run while emails differ only in case and names the clashing logins. Rolling back the metadata migration drops only that column. |
|
||||
| `lagoon.QueueMigrations` | Migrates River's schema to `lagoon.RiverSchemaVersion` and creates the `summer_jobs` job record table. |
|
||||
|
||||
@@ -201,3 +201,79 @@ func Status(gdb *gorm.DB, plugins []party.Plugin) ([]StatusRow, error) {
|
||||
}
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
// MigrationRef names one migration: the plugin (or framework set) that
|
||||
// declares it and its ID.
|
||||
type MigrationRef struct {
|
||||
Plugin string
|
||||
ID string
|
||||
}
|
||||
|
||||
// migrationSet is one history table's declared migrations.
|
||||
type migrationSet struct {
|
||||
id string
|
||||
migrations []*gormigrate.Migration
|
||||
}
|
||||
|
||||
// migrationSets lists the sets Migrate applies, in its order: the framework
|
||||
// sets (system files, backend admin, deferred bindings, job queue), then each
|
||||
// plugin's HasMigrations set in plugin order.
|
||||
func migrationSets(gdb *gorm.DB, plugins []party.Plugin) ([]migrationSet, error) {
|
||||
sqlDB, err := gdb.DB()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("lagoon: migration sets: %w", err)
|
||||
}
|
||||
sets := []migrationSet{
|
||||
{"summercms.attach", attach.Migrations},
|
||||
{"summercms.cabana", BackendAdminMigrations},
|
||||
{DeferredHistoryID, DeferredBindingMigrations},
|
||||
{QueueHistoryID, QueueMigrations(sqlDB)},
|
||||
}
|
||||
for _, p := range plugins {
|
||||
hm, ok := p.(pact.HasMigrations)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if ms := hm.Migrations(); len(ms) > 0 {
|
||||
sets = append(sets, migrationSet{p.ID(), ms})
|
||||
}
|
||||
}
|
||||
return sets, nil
|
||||
}
|
||||
|
||||
// PendingMigrations returns the migrations Migrate would apply: each set's
|
||||
// declared IDs that its history table does not hold, framework sets first,
|
||||
// then plugins in order, each in declaration order. A missing history table
|
||||
// means every migration of that set is pending. It only reads.
|
||||
func PendingMigrations(gdb *gorm.DB, plugins []party.Plugin) ([]MigrationRef, error) {
|
||||
if gdb == nil {
|
||||
return nil, fmt.Errorf("lagoon: gorm db is nil")
|
||||
}
|
||||
sets, err := migrationSets(gdb, plugins)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var pending []MigrationRef
|
||||
for _, s := range sets {
|
||||
table, err := HistoryTableName(s.id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
applied := map[string]bool{}
|
||||
if gdb.Migrator().HasTable(table) {
|
||||
var ids []string
|
||||
if err := gdb.Table(table).Pluck("id", &ids).Error; err != nil {
|
||||
return nil, fmt.Errorf("lagoon: pending %s: %w", s.id, err)
|
||||
}
|
||||
for _, id := range ids {
|
||||
applied[id] = true
|
||||
}
|
||||
}
|
||||
for _, m := range s.migrations {
|
||||
if m != nil && !applied[m.ID] {
|
||||
pending = append(pending, MigrationRef{Plugin: s.id, ID: m.ID})
|
||||
}
|
||||
}
|
||||
}
|
||||
return pending, nil
|
||||
}
|
||||
|
||||
@@ -38,7 +38,8 @@ Use cases:
|
||||
- `surf.LocaleFromPrincipal` switches the request locale to the signed-in user's preferred locale.
|
||||
- Opt-in uploads route: with `storage.uploads.serve` true, `surf.BuildRouter` mounts `GET <storage.uploads.public_path_prefix>/{path...}` (owned by `summercms.surf`) on `attach.StaticHandlerPublic`, so a file is served only when its `system_files` row is public. Every response carries `X-Content-Type-Options: nosniff` and a sandboxing `Content-Security-Policy`, and a file whose type is not a raster image (JPEG, PNG, GIF, WebP, AVIF) is sent with `Content-Disposition: attachment`, so a stored HTML or SVG file never renders on the app origin. The bucket and the database are looked up per request, so `route:list` lists the route without opening either; until `serve` publishes them it answers 404. The route owns the whole prefix: `surf.BuildRouter` fails when a plugin route sits at or under it, and when the prefix is not an absolute URL path (no leading slash, a scheme or host, a query, an empty, `.` or `..` segment, or `{`/`}`).
|
||||
- Opt-in API documentation page: with `http.api_docs.enabled` true, `surf.BuildRouter` mounts `GET /api-docs` (a Swagger UI page) and `GET /api-docs.json` (owned by `summercms.apidocs`, public, no middleware). The spec comes from the one plugin implementing `surf.APIDocsProvider`, as YAML or JSON, converted once at boot to compact JSON in source key order; zero or several providers, or a spec that does not convert, fail boot. The page loads swagger-ui-dist `surf.SwaggerUIVersion` from jsDelivr with Subresource Integrity hashes on all three assets and a `Content-Security-Policy` whose only script source is a per-request nonce; every script tag, the two jsDelivr scripts included, carries it, so no other jsDelivr script can run.
|
||||
- Opt-in request log: with `http.access_log` true, every request writes one Info record `http request` after it is served, with `method`, `pattern` (the matched route, such as `GET /posts/{id}`), `status` and `duration_ms`. A request whose client went away adds `client_gone=true`; its 5xx is logged as 499 (client closed request) only when the client had already gone when the status was written and the handler did not panic, so a 5xx written before the cancellation and a recovered panic keep their real status. An unmatched request (a 404 or 405 from the mux, or a CORS preflight answered before routing) has an empty `pattern`. The URL path, the query string, headers and the client address are never logged, for an unmatched request too: a token route requested with another method answers 405 before any route handler runs. Records go to the app's published `*slog.Logger`, else `slog.Default`, so the [sunscreen](../sunscreen/README.md) redaction still applies.
|
||||
- Request log, on by default: every request writes one Info record `http request` from the [diary](../diary/README.md) component `http` after it is served, with `method`, `pattern` (the matched route, such as `GET /posts/{id}`), `status`, `duration_ms` and `request_id` (the `X-Request-Id` header, trimmed and cut to 128 bytes, left out when empty). A request whose client went away adds `client_gone=true`; its 5xx is logged as 499 (client closed request) only when the client had already gone when the status was written and the handler did not panic, so a 5xx written before the cancellation and a recovered panic keep their real status. An unmatched request (a 404 or 405 from the mux, or a CORS preflight answered before routing) has an empty `pattern`. The URL path, the query string, headers and the client address are never logged, for an unmatched request too: a token route requested with another method answers 405 before any route handler runs. Records go to the app's published `*slog.Logger`, else the `http` component logger, whose channels all redact through [sunscreen](../sunscreen/README.md). Silence it with `log.components.http: warn` or turn it off with `http.access_log: false`.
|
||||
- Startup summary and shutdown report for `serve`: once it listens, a block on a terminal (version, environment, URL and admin URL, database host and name, pending migrations, plugins, queue, scheduler, mail, cache, realtime and search drivers; `-v` adds one line per plugin) and, off a terminal, the same facts as a `startup summary` record and a `listening on <addr>` record of component `serve`; after SIGINT or SIGTERM, a report of requests served, drained and aborted, worker jobs, uptime and drain errors. Never a DSN user or password.
|
||||
- A read-only route table (`surf.Router.Routes`), the admin mount status (`surf.Router.AdminPrefix`) and the `serve` and `route:list` commands.
|
||||
|
||||
## Usage
|
||||
@@ -144,7 +145,7 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur
|
||||
| `surf.LocaleFromPrincipal` | Middleware that applies the signed-in user's preferred locale. |
|
||||
| `surf.ErrorPage` | What a browser error page shows: status, title, message, home link, and (only with `ShowDetails`) method, path and request id. |
|
||||
| `surf.ErrorPageRenderer` | Optional backpack-published override of the HTML error page; `RenderErrorPage` returns the full document, or `ok=false` to keep the built-in page. |
|
||||
| `surf.ServeCommand` | The `serve` console command. |
|
||||
| `surf.ServeCommand` | The `serve` console command, with its startup summary, request counter and shutdown report. |
|
||||
| `surf.RouteListCommand` | The `route:list` console command. |
|
||||
|
||||
## Configuration
|
||||
@@ -171,7 +172,7 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur
|
||||
| `http.api_docs.enabled` | `false` | Mounts `GET <http.api_docs.path>` (Swagger UI page) and `GET <http.api_docs.path>.json` (the spec of the one `surf.APIDocsProvider` plugin). Both are public. Environment: `SUMMER_HTTP__API_DOCS__ENABLED`. |
|
||||
| `http.api_docs.path` | `/api-docs` | Page path; the JSON route adds `.json`. Must be an absolute URL path; a plugin route on either path fails boot. |
|
||||
| `http.api_docs.title` | `API documentation` | Page title. |
|
||||
| `http.access_log` | `false` | Logs one `http request` record per request (method, matched pattern, status, duration in milliseconds; never the path or query). Environment: `SUMMER_HTTP__ACCESS_LOG`. |
|
||||
| `http.access_log` | `true` | Logs one `http request` record per request under the `http` component (method, matched pattern, status, duration in milliseconds, request id; never the path or query). Set to `false` to turn it off, or raise `log.components.http` to `warn` to silence it. Environment: `SUMMER_HTTP__ACCESS_LOG`. |
|
||||
| `http.error_pages.enabled` | `true` | Answers browser navigations that hit a framework-generated 404, 405 or 500 with the HTML error page (or a published `surf.ErrorPageRenderer`'s page). `false` restores the plain stdlib and JSON bodies for every request. Environment: `SUMMER_HTTP__ERROR_PAGES__ENABLED`. |
|
||||
| `storage.uploads.serve` | `false` | Mounts the uploads route `GET <storage.uploads.public_path_prefix>/{path...}` (prefix default `/storage/uploads`; it must then be an absolute URL path, or start-up fails) on `attach.StaticHandlerPublic` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Prefer a separate origin for uploads in production. Environment: `SUMMER_STORAGE__UPLOADS__SERVE`. |
|
||||
|
||||
@@ -195,15 +196,15 @@ The `serve` command also opens the database through [lagoon](../lagoon/README.md
|
||||
|
||||
| Command | Flags | Description |
|
||||
|---------|-------|-------------|
|
||||
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, installs cooler (the shared rate-limit store when `http.rate_limit.store` is `redis`), assembles the router, binds the address, starts the in-process job worker (see `queue.work_in_serve`) and serves HTTP until SIGINT or SIGTERM, then shuts the server and the worker down gracefully within 10 seconds. After binding it prints the listen address with a browser URL (an unspecified host shows as 127.0.0.1) and an admin status line: the admin URL, with a warning naming `summer admin:create` when no administrator exists yet, or `admin: off` with the steps to add an admin controller. |
|
||||
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, installs cooler (the shared rate-limit store when `http.rate_limit.store` is `redis`), assembles the router, binds the address, starts the in-process job worker (see `queue.work_in_serve`) and serves HTTP until SIGINT or SIGTERM. Once it listens it prints the startup summary on a terminal (unless `-q`), with the admin URL and a warning naming `summer admin:create` when no administrator exists yet, or `off` with the steps to add an admin controller; off a terminal it logs the summary and `listening on <addr>` as records. On a signal it drains requests and the worker within 10 seconds, prints and logs the shutdown report, and exits 0, or 1 when draining fails. |
|
||||
| `route:list` | none | Builds the router the same way `serve` does, without opening the database or listening, and prints a table of method, pattern, plugin, middleware and raw flag for every route, followed by the admin status line (`admin: mounted at <prefix>` or `admin: off`). |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [cabana](../cabana/README.md), [compass](../compass/README.md), [conga](../conga/README.md) (the in-process job worker of `serve`), [cooler](../cooler/README.md) (the shared rate-limit store `serve` installs), [lagoon](../lagoon/README.md) (including `lagoon/attach`), [pact](../pact/README.md), [party](../party/README.md), [towel](../towel/README.md), [wire](../wire/README.md).
|
||||
- Third-party: `gocloud.dev/blob` (uploads bucket opened by `serve`), `github.com/goccy/go-yaml` (api-docs spec conversion).
|
||||
- Standard library: `bytes`, `context`, `crypto/rand`, `embed`, `encoding/base64`, `encoding/json`, `errors`, `fmt`, `html/template`, `log/slog`, `math`, `mime`, `net`, `net/http`, `net/netip`, `net/url`, `os`, `os/signal`, `regexp`, `sort`, `strconv`, `strings`, `sync`, `syscall`, `time`, `unicode/utf8`.
|
||||
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Redis container).
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [cabana](../cabana/README.md), [compass](../compass/README.md), [conga](../conga/README.md) (the in-process job worker of `serve`), [cooler](../cooler/README.md) (the shared rate-limit store `serve` installs), [diary](../diary/README.md) (the `http` and `serve` component loggers), [lagoon](../lagoon/README.md) (including `lagoon/attach`), [pact](../pact/README.md), [party](../party/README.md), [towel](../towel/README.md), [wire](../wire/README.md).
|
||||
- Third-party: `gocloud.dev/blob` (uploads bucket opened by `serve`), `github.com/goccy/go-yaml` (api-docs spec conversion), `github.com/jackc/pgx/v5/pgconn` (the database host and name in the startup summary).
|
||||
- Standard library: `bytes`, `context`, `crypto/rand`, `embed`, `encoding/base64`, `encoding/json`, `errors`, `fmt`, `html/template`, `log/slog`, `math`, `mime`, `net`, `net/http`, `net/netip`, `net/url`, `os`, `os/signal`, `path/filepath`, `regexp`, `sort`, `strconv`, `strings`, `sync`, `sync/atomic`, `syscall`, `time`, `unicode/utf8`.
|
||||
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Redis and Postgres containers).
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -211,4 +212,4 @@ The `serve` command also opens the database through [lagoon](../lagoon/README.md
|
||||
go test ./modules/surf/...
|
||||
```
|
||||
|
||||
The tests use `net/http/httptest` and in-memory stores. `TestRateLimitAcrossInstancesSmoke` starts a Redis container through testcontainers-go and needs Docker; `go test -short ./modules/surf/...` skips it.
|
||||
The tests use `net/http/httptest` and in-memory stores. `TestRateLimitAcrossInstancesSmoke` starts a Redis container and `TestServeLifecycle` a Postgres container (it runs `serve`, sends one request and stops it) through testcontainers-go; both need Docker, and `go test -short ./modules/surf/...` skips them. `TestServeSummarySmoke` and `TestShutdownReportSmoke` check the summary and report builders without a database.
|
||||
|
||||
@@ -8,11 +8,16 @@ import (
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/diary"
|
||||
)
|
||||
|
||||
// accessLogKey is the config key that turns the request log on.
|
||||
// accessLogKey is the config key that turns the request log off (false).
|
||||
const accessLogKey = "http.access_log"
|
||||
|
||||
// httpLog is the request log's component logger (D-23): silence it with
|
||||
// log.components.http: warn or http.access_log: false.
|
||||
var httpLog = diary.For("http")
|
||||
|
||||
// accessLogMessage is the message of every request log record.
|
||||
const accessLogMessage = "http request"
|
||||
|
||||
@@ -34,14 +39,19 @@ type accessEntryKey struct{}
|
||||
// had gone away, when the error came from that cancellation.
|
||||
const statusClientClosedRequest = 499
|
||||
|
||||
// accessLogEnabled reports whether http.access_log is true.
|
||||
// accessLogEnabled reports whether the request log is on: it is unless
|
||||
// http.access_log is set to false.
|
||||
func accessLogEnabled(app *backpack.App) bool {
|
||||
return app != nil && app.Config != nil && app.Config.Bool(accessLogKey)
|
||||
if app == nil || app.Config == nil || !app.Config.Has(accessLogKey) {
|
||||
return true
|
||||
}
|
||||
return app.Config.Bool(accessLogKey)
|
||||
}
|
||||
|
||||
// accessLog logs one Info record "http request" per request after it is
|
||||
// served, with method, pattern (the matched route, "" when none matched),
|
||||
// status and duration_ms (whole milliseconds). The URL path is never
|
||||
// status, duration_ms (whole milliseconds) and request_id (the X-Request-Id
|
||||
// header trimmed and cut to 128 bytes, left out when empty). The URL path is never
|
||||
// logged, not even for a request that matched no route: a ServeMux 405 for
|
||||
// a token route reached with another method, or a 404 for its trailing-slash
|
||||
// variant, would otherwise carry the token, because share and invitation
|
||||
@@ -55,9 +65,10 @@ func accessLogEnabled(app *backpack.App) bool {
|
||||
// failed because of the cancellation, and no client received the error. A
|
||||
// 5xx written before the client went away keeps its status, and a recovered
|
||||
// panic is always 500, so a search for 5xx records still finds every real
|
||||
// server error. The logger is the app's published *slog.Logger, else
|
||||
// slog.Default (which the generated main wraps in the redacting sunscreen
|
||||
// handler).
|
||||
// server error. The logger is the app's published *slog.Logger, else the
|
||||
// diary component logger http, whose channels all redact through sunscreen.
|
||||
// The route pattern, never the raw path, is what D-23 settled on: the log
|
||||
// is on by default, so it must be safe to keep in journald and daily files.
|
||||
func accessLog(app *backpack.App, next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
|
||||
start := time.Now()
|
||||
@@ -77,6 +88,9 @@ func accessLog(app *backpack.App, next http.Handler) http.Handler {
|
||||
slog.Int("status", status),
|
||||
slog.Int64("duration_ms", time.Since(start).Milliseconds()),
|
||||
}
|
||||
if id := requestID(req); id != "" {
|
||||
attrs = append(attrs, slog.String("request_id", id))
|
||||
}
|
||||
if clientGone(req.Context()) {
|
||||
attrs = append(attrs, slog.Bool("client_gone", true))
|
||||
}
|
||||
@@ -97,14 +111,14 @@ func markPanicked(req *http.Request) {
|
||||
}
|
||||
}
|
||||
|
||||
// accessLogger is the app's published logger, else slog.Default.
|
||||
// accessLogger is the app's published logger, else httpLog.
|
||||
func accessLogger(app *backpack.App) *slog.Logger {
|
||||
if app != nil {
|
||||
if log, ok := app.Lookup[*slog.Logger](); ok && log != nil {
|
||||
return log
|
||||
}
|
||||
}
|
||||
return slog.Default()
|
||||
return httpLog
|
||||
}
|
||||
|
||||
// recordPattern stores pattern in the request's access log entry, when the
|
||||
|
||||
@@ -33,13 +33,14 @@ func (blogPostsPlugin) Routes(r pact.Router) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// accessLogRouter assembles the blog plugin with http.access_log set to on
|
||||
// and a published JSON logger writing into the returned buffer.
|
||||
// accessLogRouter assembles the blog plugin with http.access_log left at
|
||||
// its default (on) or set to false, and a published JSON logger writing
|
||||
// into the returned buffer.
|
||||
func accessLogRouter(t *testing.T, on bool) (http.Handler, *bytes.Buffer) {
|
||||
t.Helper()
|
||||
y := "body_limits:\n default_bytes: 1024\n upload_bytes: 1024\n"
|
||||
if on {
|
||||
y += "access_log: true\n"
|
||||
if !on {
|
||||
y += "access_log: false\n"
|
||||
}
|
||||
app := backpack.New(writeHTTPConfig(t, y))
|
||||
var buf bytes.Buffer
|
||||
@@ -76,18 +77,23 @@ func TestAccessLogSmoke(t *testing.T) {
|
||||
h, buf := accessLogRouter(t, true)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/blog/posts/7?token=secret", nil))
|
||||
req := httptest.NewRequest(http.MethodGet, "/blog/posts/7?token=secret", nil)
|
||||
req.Header.Set("X-Request-Id", " req-42 ")
|
||||
h.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("GET /blog/posts/7 = %d", rec.Code)
|
||||
}
|
||||
recs := accessRecords(t, buf)
|
||||
if len(recs) != 1 {
|
||||
t.Fatalf("records = %d, want 1:\n%s", len(recs), buf.String())
|
||||
t.Fatalf("records = %d, want 1 (the request log is on by default):\n%s", len(recs), buf.String())
|
||||
}
|
||||
r := recs[0]
|
||||
if r["method"] != "GET" || r["pattern"] != "GET /blog/posts/{id}" || r["status"] != float64(200) || r["level"] != "INFO" {
|
||||
t.Fatalf("matched record = %v", r)
|
||||
}
|
||||
if r["request_id"] != "req-42" {
|
||||
t.Fatalf("request_id = %v, want the trimmed X-Request-Id", r["request_id"])
|
||||
}
|
||||
if _, ok := r["duration_ms"].(float64); !ok {
|
||||
t.Fatalf("matched record has no duration_ms: %v", r)
|
||||
}
|
||||
@@ -112,6 +118,17 @@ func TestAccessLogSmoke(t *testing.T) {
|
||||
if r["pattern"] != "" || r["status"] != float64(404) {
|
||||
t.Fatalf("unmatched record = %v", r)
|
||||
}
|
||||
if _, ok := r["request_id"]; ok {
|
||||
t.Fatalf("a request without X-Request-Id has a request_id: %v", r)
|
||||
}
|
||||
|
||||
buf.Reset()
|
||||
long := httptest.NewRequest(http.MethodGet, "/blog/posts/7", nil)
|
||||
long.Header.Set("X-Request-Id", strings.Repeat("x", 300))
|
||||
h.ServeHTTP(httptest.NewRecorder(), long)
|
||||
if recs = accessRecords(t, buf); len(recs) != 1 || len(recs[0]["request_id"].(string)) != errorPageMaxRequestID {
|
||||
t.Fatalf("a long X-Request-Id is not cut to %d bytes: %v", errorPageMaxRequestID, recs)
|
||||
}
|
||||
if _, ok := r["path"]; ok {
|
||||
t.Fatalf("unmatched record carries path: %v", r)
|
||||
}
|
||||
@@ -242,6 +259,8 @@ func accessLogEdgesRouter(t *testing.T, on, locale bool, flushErr *error) (http.
|
||||
y := "body_limits:\n default_bytes: 1024\n upload_bytes: 1024\n"
|
||||
if on {
|
||||
y += "access_log: true\n"
|
||||
} else {
|
||||
y += "access_log: false\n"
|
||||
}
|
||||
app := backpack.New(writeHTTPConfig(t, y))
|
||||
var buf bytes.Buffer
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"net/netip"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
@@ -17,6 +18,7 @@ import (
|
||||
"git.golem15.com/golem15/summercms/modules/cabana"
|
||||
"git.golem15.com/golem15/summercms/modules/conga"
|
||||
"git.golem15.com/golem15/summercms/modules/cooler"
|
||||
"git.golem15.com/golem15/summercms/modules/diary"
|
||||
"git.golem15.com/golem15/summercms/modules/lagoon"
|
||||
"git.golem15.com/golem15/summercms/modules/lagoon/attach"
|
||||
"git.golem15.com/golem15/summercms/modules/party"
|
||||
@@ -25,16 +27,40 @@ import (
|
||||
)
|
||||
|
||||
// ServeCommand starts a signal-aware HTTP server on the assembled router.
|
||||
// Once it listens it prints a startup summary on a terminal and logs it as
|
||||
// records (component serve); on SIGINT or SIGTERM it drains requests and the
|
||||
// in-process worker for up to 10 seconds, reports what it did and returns
|
||||
// nil, or the drain errors.
|
||||
func ServeCommand(app *backpack.App, plugins []party.Plugin) bonfire.Command {
|
||||
return bonfire.Command{
|
||||
Name: "serve",
|
||||
Description: "Serve the application HTTP API",
|
||||
Long: `Opens the database and the uploads bucket, installs the cache, assembles every
|
||||
plugin's routes, binds the address and starts the in-process job worker
|
||||
(queue.work_in_serve). Once it listens it prints a startup summary: version,
|
||||
environment, URL and admin URL, database and pending migrations, plugins, and
|
||||
the queue, scheduler, mail, cache, realtime and search drivers. -v adds one
|
||||
line per plugin; -q prints only warnings and errors. Off a terminal the same
|
||||
facts are logged as a "startup summary" record and a "listening on <addr>"
|
||||
record, so journald and the log files keep them.
|
||||
|
||||
Each request is logged as an "http request" record of component http (route
|
||||
pattern, never the path). On Ctrl+C or SIGTERM serve drains requests and the
|
||||
worker for up to 10 seconds and prints a short report. It exits 0 after a
|
||||
clean stop and 1 when draining fails.`,
|
||||
Flags: []bonfire.Flag{{
|
||||
Name: "addr",
|
||||
Description: "Listen address",
|
||||
Default: ":8080",
|
||||
}},
|
||||
Examples: []bonfire.Example{
|
||||
{Command: "serve", Description: "Listen on every interface, port 8080"},
|
||||
{Command: "serve --addr 127.0.0.1:8096", Description: "Listen on loopback only, during development"},
|
||||
{Command: "serve -v", Description: "Add one summary line per plugin and info logs of every component"},
|
||||
{Command: "serve -q", Description: "Print only warnings and errors"},
|
||||
},
|
||||
Run: func(ctx context.Context, in bonfire.Input, out bonfire.Output) error {
|
||||
started := time.Now()
|
||||
addr, _ := in.Flag("addr")
|
||||
if strings.TrimSpace(addr) == "" {
|
||||
addr = ":8080"
|
||||
@@ -67,6 +93,7 @@ func ServeCommand(app *backpack.App, plugins []party.Plugin) bonfire.Command {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
counter := newRequestCounter(h)
|
||||
ln, err := net.Listen("tcp", addr)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -76,16 +103,33 @@ func ServeCommand(app *backpack.App, plugins []party.Plugin) bonfire.Command {
|
||||
_ = ln.Close()
|
||||
return err
|
||||
}
|
||||
if worker != nil {
|
||||
out.Info(fmt.Sprintf("job worker started on queues: %s", strings.Join(worker.Queues(), ", ")))
|
||||
}
|
||||
prefix, mounted := r.AdminPrefix()
|
||||
admins := adminsUnknown
|
||||
if mounted {
|
||||
admins = countAdmins(ctx, gdb)
|
||||
}
|
||||
printBanner(out, serveBanner(addr, prefix, mounted, admins))
|
||||
srv := &http.Server{Addr: addr, Handler: h, BaseContext: func(net.Listener) context.Context { return ctx }}
|
||||
pending, pendingErr := lagoon.PendingMigrations(gdb, plugins)
|
||||
facts := buildFacts(app, factInputs{
|
||||
addr: listenAddr(addr, ln),
|
||||
bin: binName(),
|
||||
plugins: plugins,
|
||||
queues: worker.Queues(),
|
||||
worker: worker != nil,
|
||||
adminPrefix: prefix,
|
||||
mounted: mounted,
|
||||
admins: admins,
|
||||
pending: pending,
|
||||
pendingErr: pendingErr,
|
||||
version: bonfire.BuildVersion(),
|
||||
})
|
||||
styled := bonfire.IsTerminal(out) && !bonfire.IsQuiet(out)
|
||||
logCtx := ctx
|
||||
if styled {
|
||||
printSummary(out, facts)
|
||||
logCtx = diary.HideOnTTY(ctx)
|
||||
}
|
||||
logSummary(logCtx, serveLog, facts)
|
||||
srv := &http.Server{Addr: addr, Handler: counter, BaseContext: func(net.Listener) context.Context { return ctx }}
|
||||
errCh := make(chan error, 1)
|
||||
go func() {
|
||||
errCh <- srv.Serve(ln)
|
||||
@@ -95,18 +139,36 @@ func ServeCommand(app *backpack.App, plugins []party.Plugin) bonfire.Command {
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
report := shutdownReport{Draining: counter.inFlight.Load()}
|
||||
if styled {
|
||||
out.Info(fmt.Sprintf("Shutting down: draining %d requests", report.Draining))
|
||||
}
|
||||
sctx, cancel := shutdownCtx()
|
||||
defer cancel()
|
||||
shutdownErr := srv.Shutdown(sctx)
|
||||
workerErr := stopWorker(sctx, worker)
|
||||
if shutdownErr != nil {
|
||||
return shutdownErr
|
||||
if err := srv.Shutdown(sctx); err != nil {
|
||||
report.Aborted = counter.inFlight.Load()
|
||||
_ = srv.Close()
|
||||
report.Errors = append(report.Errors, fmt.Errorf("surf: drain requests: %w", err))
|
||||
}
|
||||
err := <-errCh
|
||||
if err != nil && err != http.ErrServerClosed {
|
||||
return err
|
||||
if err := stopWorker(sctx, worker); err != nil {
|
||||
report.Errors = append(report.Errors, err)
|
||||
}
|
||||
return workerErr
|
||||
if err := <-errCh; err != nil && err != http.ErrServerClosed {
|
||||
report.Errors = append(report.Errors, err)
|
||||
}
|
||||
if worker != nil {
|
||||
stats := worker.Stats()
|
||||
report.Worker = &stats
|
||||
}
|
||||
report.Served = counter.served.Load()
|
||||
report.Uptime = time.Since(started)
|
||||
reportCtx := context.WithoutCancel(ctx)
|
||||
if styled {
|
||||
bonfire.Details(out, "Shutdown report", report.details())
|
||||
reportCtx = diary.HideOnTTY(reportCtx)
|
||||
}
|
||||
logReport(reportCtx, serveLog, report)
|
||||
return report.Err()
|
||||
case err := <-errCh:
|
||||
sctx, cancel := shutdownCtx()
|
||||
defer cancel()
|
||||
@@ -120,6 +182,20 @@ func ServeCommand(app *backpack.App, plugins []party.Plugin) bonfire.Command {
|
||||
}
|
||||
}
|
||||
|
||||
// listenAddr is addr, or the bound address when addr asks for any port.
|
||||
func listenAddr(addr string, ln net.Listener) string {
|
||||
if _, port, err := net.SplitHostPort(addr); err == nil && port == "0" && ln != nil {
|
||||
return ln.Addr().String()
|
||||
}
|
||||
return addr
|
||||
}
|
||||
|
||||
// binName is how a person runs this binary from the application root:
|
||||
// bin/<name of the running executable>.
|
||||
func binName() string {
|
||||
return "bin/" + filepath.Base(os.Args[0])
|
||||
}
|
||||
|
||||
// stopWorker stops the in-process job worker, if serve started one.
|
||||
func stopWorker(ctx context.Context, w *conga.Worker) error {
|
||||
if w == nil {
|
||||
@@ -232,44 +308,39 @@ func countAdmins(ctx context.Context, gdb *gorm.DB) adminPresence {
|
||||
return adminsPresent
|
||||
}
|
||||
|
||||
// bannerLine is one startup line; warn lines print with the warning symbol.
|
||||
// bannerLine is one startup note; warn notes print with the warning symbol.
|
||||
type bannerLine struct {
|
||||
warn bool
|
||||
text string
|
||||
}
|
||||
|
||||
// serveBanner builds the lines serve prints once the port is bound: the
|
||||
// listen address with a browser URL, then the admin status.
|
||||
func serveBanner(addr, adminPrefix string, mounted bool, admins adminPresence) []bannerLine {
|
||||
url := serveURL(addr)
|
||||
listen := "listening on " + addr
|
||||
if url != "" {
|
||||
listen += ", open " + url
|
||||
}
|
||||
lines := []bannerLine{{text: listen}}
|
||||
// adminStatus is the summary's Admin value and the notes printed after the
|
||||
// summary: the admin URL (url is serve's browser URL, "" when the address
|
||||
// does not parse) with a warning naming summer admin:create when no
|
||||
// administrator exists yet, or "off" with the steps to add an admin
|
||||
// controller.
|
||||
func adminStatus(url, adminPrefix string, mounted bool, admins adminPresence) (string, []bannerLine) {
|
||||
if !mounted {
|
||||
return append(lines,
|
||||
bannerLine{text: adminOffText},
|
||||
bannerLine{text: "to add one, run summer make:admin-controller <vendor.plugin> <Name>, then summer build, summer migrate and " + adminCreateCommand + "; the admin needs SUMMER_ADMIN__JWT__SECRET, which summer new writes to .env"},
|
||||
)
|
||||
return strings.TrimPrefix(adminOffText, "admin: "), []bannerLine{
|
||||
{text: "to add an admin, run summer make:admin-controller <vendor.plugin> <Name>, then summer build, summer migrate and " + adminCreateCommand + "; the admin needs SUMMER_ADMIN__JWT__SECRET, which summer new writes to .env"},
|
||||
}
|
||||
}
|
||||
value := "mounted at " + adminPrefix
|
||||
if url != "" {
|
||||
lines = append(lines, bannerLine{text: "admin: " + strings.TrimSuffix(url, "/") + adminPrefix})
|
||||
} else {
|
||||
lines = append(lines, bannerLine{text: "admin: mounted at " + adminPrefix})
|
||||
value = strings.TrimSuffix(url, "/") + adminPrefix
|
||||
}
|
||||
switch admins {
|
||||
case adminsNone:
|
||||
lines = append(lines, bannerLine{warn: true, text: "no administrators yet; create one with " + adminCreateCommand})
|
||||
return value, []bannerLine{{warn: true, text: "no administrators yet; create one with " + adminCreateCommand}}
|
||||
case adminsMissingTable:
|
||||
lines = append(lines, bannerLine{warn: true, text: "the admin tables do not exist yet; run summer migrate, then " + adminCreateCommand})
|
||||
return value, []bannerLine{{warn: true, text: "the admin tables do not exist yet; run summer migrate, then " + adminCreateCommand}}
|
||||
case adminsUnknown:
|
||||
lines = append(lines, bannerLine{text: "sign in with an administrator; " + adminCreateCommand + " creates one"})
|
||||
return value, []bannerLine{{text: "sign in with an administrator; " + adminCreateCommand + " creates one"}}
|
||||
}
|
||||
return lines
|
||||
return value, nil
|
||||
}
|
||||
|
||||
// printBanner writes banner lines as info or warning output.
|
||||
// printBanner writes notes as info or warning output.
|
||||
func printBanner(out bonfire.Output, lines []bannerLine) {
|
||||
for _, l := range lines {
|
||||
if l.warn {
|
||||
|
||||
198
modules/surf/serve_lifecycle_test.go
Normal file
198
modules/surf/serve_lifecycle_test.go
Normal file
@@ -0,0 +1,198 @@
|
||||
package surf
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/bonfire"
|
||||
"git.golem15.com/golem15/summercms/modules/compass"
|
||||
"git.golem15.com/golem15/summercms/modules/diary"
|
||||
"git.golem15.com/golem15/summercms/modules/lagoon"
|
||||
"git.golem15.com/golem15/summercms/modules/pact"
|
||||
"git.golem15.com/golem15/summercms/modules/party"
|
||||
"github.com/testcontainers/testcontainers-go"
|
||||
"github.com/testcontainers/testcontainers-go/modules/postgres"
|
||||
)
|
||||
|
||||
// lifecyclePlugin serves GET /probe/items/{id}.
|
||||
type lifecyclePlugin struct{}
|
||||
|
||||
func (lifecyclePlugin) ID() string { return "acme.probe" }
|
||||
func (lifecyclePlugin) Requires() []string { return nil }
|
||||
func (lifecyclePlugin) Register(*backpack.App) error { return nil }
|
||||
func (lifecyclePlugin) Boot(*backpack.App) error { return nil }
|
||||
func (lifecyclePlugin) Routes(r pact.Router) error {
|
||||
r.Get("/probe/items/{id}", func(w http.ResponseWriter, _ *http.Request) {
|
||||
_, _ = w.Write([]byte("ok"))
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
// syncBuffer is a bytes.Buffer safe for the log writes of serve's
|
||||
// goroutines and the test's reads.
|
||||
type syncBuffer struct {
|
||||
mu sync.Mutex
|
||||
buf bytes.Buffer
|
||||
}
|
||||
|
||||
func (b *syncBuffer) Write(p []byte) (int, error) {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
return b.buf.Write(p)
|
||||
}
|
||||
|
||||
func (b *syncBuffer) String() string {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
return b.buf.String()
|
||||
}
|
||||
|
||||
// TestServeLifecycle runs serve against Postgres: it binds, logs the
|
||||
// startup summary and "listening on <addr>", logs a request by its route
|
||||
// pattern under component http with its request id, and on cancellation
|
||||
// drains, logs the shutdown report and returns nil (exit 0).
|
||||
func TestServeLifecycle(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("requires testcontainers postgres")
|
||||
}
|
||||
ctx := t.Context()
|
||||
ctr, err := postgres.Run(ctx, "postgres:16-alpine",
|
||||
postgres.WithDatabase("surf"),
|
||||
postgres.WithUsername("surf"),
|
||||
postgres.WithPassword("surf-pass"),
|
||||
postgres.BasicWaitStrategies(),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("testcontainers postgres: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = testcontainers.TerminateContainer(ctr) })
|
||||
dsn, err := ctr.ConnectionString(ctx, "sslmode=disable")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
ln, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
addr := ln.Addr().String()
|
||||
_ = ln.Close()
|
||||
|
||||
dir := t.TempDir()
|
||||
files := map[string]string{
|
||||
"app.yaml": "name: probe\n",
|
||||
"database.yaml": fmt.Sprintf("dsn: %q\n", dsn),
|
||||
"storage.yaml": fmt.Sprintf("uploads:\n bucket_url: %q\n", "file://"+t.TempDir()),
|
||||
}
|
||||
for name, body := range files {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
cfg, err := compass.Open(compass.Options{Dir: dir, Env: "development", Environ: []string{"SUMMER_ENV=development"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
app := backpack.New(cfg)
|
||||
plugins := []party.Plugin{lifecyclePlugin{}}
|
||||
|
||||
// Migrate first so the in-process worker finds River's tables.
|
||||
sqlDB, gdb, err := lagoon.Open(ctx, dsn)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := lagoon.Migrate(gdb, plugins); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = sqlDB.Close()
|
||||
|
||||
var logs syncBuffer
|
||||
diary.Bootstrap(&logs, diary.Options{})
|
||||
t.Cleanup(diary.Reset)
|
||||
|
||||
runCtx, cancel := context.WithCancel(ctx)
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
var out bytes.Buffer
|
||||
go func() {
|
||||
done <- bonfire.Call(runCtx, []bonfire.Command{ServeCommand(app, plugins)}, "serve", []string{"--addr", addr}, &out)
|
||||
}()
|
||||
|
||||
url := "http://" + addr + "/probe/items/secret-777"
|
||||
var resp *http.Response
|
||||
deadline := time.Now().Add(30 * time.Second)
|
||||
for {
|
||||
req, _ := http.NewRequest(http.MethodGet, url, nil)
|
||||
req.Header.Set("X-Request-Id", "lifecycle-1")
|
||||
resp, err = http.DefaultClient.Do(req)
|
||||
if err == nil {
|
||||
break
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
t.Fatalf("serve returned before listening: %v\n%s", err, logs.String())
|
||||
default:
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatalf("serve did not listen on %s: %v\n%s", addr, err, logs.String())
|
||||
}
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
}
|
||||
_ = resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("GET %s = %d", url, resp.StatusCode)
|
||||
}
|
||||
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Fatalf("serve returned %v after a graceful stop, want nil\n%s", err, logs.String())
|
||||
}
|
||||
case <-time.After(30 * time.Second):
|
||||
t.Fatal("serve did not stop")
|
||||
}
|
||||
|
||||
got := logs.String()
|
||||
for _, want := range []string{
|
||||
"listening on " + addr,
|
||||
"msg=\"startup summary\"",
|
||||
"migrations_pending=0",
|
||||
"msg=\"shutdown report\"",
|
||||
"served=1",
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("logs lack %q:\n%s", want, got)
|
||||
}
|
||||
}
|
||||
var requests []string
|
||||
for _, line := range strings.Split(got, "\n") {
|
||||
if strings.Contains(line, "msg=\"http request\"") {
|
||||
requests = append(requests, line)
|
||||
}
|
||||
}
|
||||
if len(requests) != 1 {
|
||||
t.Fatalf("want one http request record, got %d:\n%s", len(requests), got)
|
||||
}
|
||||
for _, want := range []string{"component=http", "pattern=\"GET /probe/items/{id}\"", "request_id=lifecycle-1", "status=200"} {
|
||||
if !strings.Contains(requests[0], want) {
|
||||
t.Errorf("request record lacks %s: %s", want, requests[0])
|
||||
}
|
||||
}
|
||||
if strings.Contains(got, "secret-777") {
|
||||
t.Fatalf("the request path reached the logs:\n%s", got)
|
||||
}
|
||||
if strings.Contains(got, "surf-pass") {
|
||||
t.Fatalf("the DSN password reached the logs:\n%s", got)
|
||||
}
|
||||
}
|
||||
235
modules/surf/serve_summary_smoke_test.go
Normal file
235
modules/surf/serve_summary_smoke_test.go
Normal file
@@ -0,0 +1,235 @@
|
||||
package surf
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/bonfire"
|
||||
"git.golem15.com/golem15/summercms/modules/compass"
|
||||
"git.golem15.com/golem15/summercms/modules/conga"
|
||||
"git.golem15.com/golem15/summercms/modules/lagoon"
|
||||
"git.golem15.com/golem15/summercms/modules/party"
|
||||
)
|
||||
|
||||
// summaryPlugin is a plugin with an ID and required plugins.
|
||||
type summaryPlugin struct {
|
||||
id string
|
||||
req []string
|
||||
}
|
||||
|
||||
func (p summaryPlugin) ID() string { return p.id }
|
||||
func (p summaryPlugin) Requires() []string { return p.req }
|
||||
func (p summaryPlugin) Register(*backpack.App) error { return nil }
|
||||
func (p summaryPlugin) Boot(*backpack.App) error { return nil }
|
||||
|
||||
// summaryApp is an app whose config holds a DSN with a user and password.
|
||||
func summaryApp(t *testing.T) *backpack.App {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
files := map[string]string{
|
||||
"app.yaml": "name: acme\n",
|
||||
"database.yaml": "dsn: postgres://summer_user:pa55-secret@db.internal:5433/acme_prod?sslmode=disable\n",
|
||||
"mail.yaml": "driver: smtp\n",
|
||||
"cache.yaml": "driver: redis\n",
|
||||
"realtime.yaml": "driver: centrifugo\n",
|
||||
}
|
||||
for name, body := range files {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
cfg, err := compass.Open(compass.Options{Dir: dir, Env: "staging", Environ: []string{"SUMMER_ENV=staging"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return backpack.New(cfg)
|
||||
}
|
||||
|
||||
func TestServeSummarySmoke(t *testing.T) {
|
||||
plugins := []party.Plugin{
|
||||
summaryPlugin{id: "acme.user"},
|
||||
summaryPlugin{id: "acme.blog", req: []string{"acme.user"}},
|
||||
}
|
||||
facts := buildFacts(summaryApp(t), factInputs{
|
||||
addr: "127.0.0.1:8096",
|
||||
bin: "bin/acme",
|
||||
plugins: plugins,
|
||||
queues: []string{"default", "mail"},
|
||||
worker: true,
|
||||
adminPrefix: "/backend",
|
||||
mounted: true,
|
||||
admins: adminsNone,
|
||||
pending: []lagoon.MigrationRef{
|
||||
{Plugin: "summercms.cabana", ID: "0001"},
|
||||
{Plugin: "acme.blog", ID: "0001_posts"},
|
||||
{Plugin: "acme.blog", ID: "0002_tags"},
|
||||
},
|
||||
version: bonfire.Version{App: "v1.4.0", Framework: "v0.4.0"},
|
||||
})
|
||||
|
||||
var block bytes.Buffer
|
||||
out := bonfire.NewOutput(strings.NewReader(""), &block, &block)
|
||||
printSummary(out, facts)
|
||||
got := block.String()
|
||||
for _, want := range []string{
|
||||
"SummerCMS acme v1.4.0 (framework v0.4.0)",
|
||||
"Environment staging",
|
||||
"http://127.0.0.1:8096/",
|
||||
"http://127.0.0.1:8096/backend",
|
||||
"db.internal:5433/acme_prod",
|
||||
"3 pending — run bin/acme migrate",
|
||||
"2 (acme.user, acme.blog)",
|
||||
"default, mail (worker in serve)",
|
||||
"Mail smtp",
|
||||
"Cache redis",
|
||||
"Realtime centrifugo",
|
||||
"Search null",
|
||||
"no administrators yet",
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("summary block lacks %q:\n%s", want, got)
|
||||
}
|
||||
}
|
||||
if strings.Contains(got, "requires acme.user") {
|
||||
t.Errorf("verbosity 0 prints per-plugin lines:\n%s", got)
|
||||
}
|
||||
verbose := facts.details(1)
|
||||
var perPlugin bool
|
||||
for _, d := range verbose {
|
||||
if strings.TrimSpace(d.Label) == "acme.blog" && strings.Contains(d.Value, "requires acme.user; 2 pending migrations") {
|
||||
perPlugin = true
|
||||
}
|
||||
}
|
||||
if !perPlugin {
|
||||
t.Errorf("verbosity 1 lacks the acme.blog line: %#v", verbose)
|
||||
}
|
||||
|
||||
var logs bytes.Buffer
|
||||
logSummary(context.Background(), slog.New(slog.NewJSONHandler(&logs, nil)), facts)
|
||||
var summary map[string]any
|
||||
listening := 0
|
||||
for _, line := range strings.Split(strings.TrimSpace(logs.String()), "\n") {
|
||||
var rec map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &rec); err != nil {
|
||||
t.Fatalf("log line %q: %v", line, err)
|
||||
}
|
||||
switch rec["msg"] {
|
||||
case summaryMessage:
|
||||
summary = rec
|
||||
case "listening on 127.0.0.1:8096":
|
||||
listening++
|
||||
}
|
||||
}
|
||||
if listening != 1 {
|
||||
t.Fatalf("want one record \"listening on 127.0.0.1:8096\":\n%s", logs.String())
|
||||
}
|
||||
if summary == nil {
|
||||
t.Fatalf("no %q record:\n%s", summaryMessage, logs.String())
|
||||
}
|
||||
for key, want := range map[string]any{
|
||||
"environment": "staging",
|
||||
"database": "db.internal:5433/acme_prod",
|
||||
"migrations_pending": float64(3),
|
||||
"plugins": float64(2),
|
||||
"plugin_ids": "acme.user,acme.blog",
|
||||
"mail": "smtp",
|
||||
"cache": "redis",
|
||||
"realtime": "centrifugo",
|
||||
"search": "null",
|
||||
} {
|
||||
if summary[key] != want {
|
||||
t.Errorf("summary record %s = %v, want %v", key, summary[key], want)
|
||||
}
|
||||
}
|
||||
for _, secret := range []string{"pa55-secret", "summer_user"} {
|
||||
if strings.Contains(got, secret) || strings.Contains(logs.String(), secret) {
|
||||
t.Fatalf("summary leaks %q:\n%s\n%s", secret, got, logs.String())
|
||||
}
|
||||
}
|
||||
if databaseFact("host=db port=5432 dbname=acme user=u password=p") != "db:5432/acme" {
|
||||
t.Errorf("keyword DSN = %q", databaseFact("host=db port=5432 dbname=acme user=u password=p"))
|
||||
}
|
||||
if databaseFact("postgres://u:p@[::1:bad") != "configured" {
|
||||
t.Errorf("unparsable DSN = %q, want configured", databaseFact("postgres://u:p@[::1:bad"))
|
||||
}
|
||||
|
||||
probe := []bonfire.Command{{
|
||||
Name: "probe:summary",
|
||||
Run: func(_ context.Context, _ bonfire.Input, out bonfire.Output) error {
|
||||
printSummary(out, facts)
|
||||
return nil
|
||||
},
|
||||
}}
|
||||
for _, tc := range []struct {
|
||||
args []string
|
||||
block bool
|
||||
}{{nil, true}, {[]string{"-q"}, false}} {
|
||||
var buf bytes.Buffer
|
||||
if err := bonfire.Call(context.Background(), probe, "probe:summary", tc.args, &buf); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(buf.String(), "Environment") != tc.block {
|
||||
t.Errorf("args %v: block printed = %v, want %v:\n%s", tc.args, !tc.block, tc.block, buf.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestShutdownReportSmoke(t *testing.T) {
|
||||
report := shutdownReport{
|
||||
Draining: 3,
|
||||
Aborted: 1,
|
||||
Served: 120,
|
||||
Worker: &conga.WorkerStats{Done: 7, Failed: 2, HandedBack: 1},
|
||||
Uptime: 3*time.Minute + 4*time.Second,
|
||||
Errors: []error{errors.New("surf: drain requests: context deadline exceeded")},
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
out := bonfire.NewOutput(strings.NewReader(""), &buf, &buf)
|
||||
bonfire.Details(out, "Shutdown report", report.details())
|
||||
got := buf.String()
|
||||
for _, want := range []string{
|
||||
"Shutdown report",
|
||||
"Requests 120 served, 2 drained, 1 aborted",
|
||||
"Worker 7 done, 2 failed, 1 handed back to the queue",
|
||||
"Uptime 3m4s",
|
||||
"Errors surf: drain requests: context deadline exceeded",
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("report lacks %q:\n%s", want, got)
|
||||
}
|
||||
}
|
||||
if report.Err() == nil {
|
||||
t.Fatal("a drain error must make the report fail")
|
||||
}
|
||||
|
||||
var logs bytes.Buffer
|
||||
logReport(context.Background(), slog.New(slog.NewJSONHandler(&logs, nil)), report)
|
||||
var rec map[string]any
|
||||
if err := json.Unmarshal(logs.Bytes(), &rec); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rec["msg"] != reportMessage || rec["level"] != "ERROR" || rec["served"] != float64(120) ||
|
||||
rec["drained"] != float64(2) || rec["aborted"] != float64(1) || rec["jobs_done"] != float64(7) ||
|
||||
rec["jobs_handed_back"] != float64(1) {
|
||||
t.Fatalf("report record = %v", rec)
|
||||
}
|
||||
|
||||
clean := shutdownReport{Served: 5, Uptime: time.Second}
|
||||
if clean.Err() != nil {
|
||||
t.Fatal("a stop without errors must exit 0")
|
||||
}
|
||||
buf.Reset()
|
||||
bonfire.Details(out, "", clean.details())
|
||||
if !strings.Contains(buf.String(), "Worker not running in serve") || !strings.Contains(buf.String(), "Errors none") {
|
||||
t.Fatalf("clean report:\n%s", buf.String())
|
||||
}
|
||||
}
|
||||
@@ -73,9 +73,9 @@ func TestServeURL(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestServeBanner checks the summary's Admin value and the notes printed
|
||||
// after the summary for each admin state.
|
||||
func TestServeBanner(t *testing.T) {
|
||||
listen := bannerLine{text: "listening on :8123, open http://127.0.0.1:8123/"}
|
||||
mounted := bannerLine{text: "admin: http://127.0.0.1:8123/backend"}
|
||||
create := "summer admin:create --email you@example.com --superuser"
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
@@ -83,25 +83,24 @@ func TestServeBanner(t *testing.T) {
|
||||
prefix string
|
||||
mounted bool
|
||||
admins adminPresence
|
||||
want []bannerLine
|
||||
value string
|
||||
notes []bannerLine
|
||||
}{
|
||||
{"present", ":8123", "/backend", true, adminsPresent, []bannerLine{listen, mounted}},
|
||||
{"none", ":8123", "/backend", true, adminsNone, []bannerLine{listen, mounted,
|
||||
{"present", ":8123", "/backend", true, adminsPresent, "http://127.0.0.1:8123/backend", nil},
|
||||
{"none", ":8123", "/backend", true, adminsNone, "http://127.0.0.1:8123/backend", []bannerLine{
|
||||
{warn: true, text: "no administrators yet; create one with " + create}}},
|
||||
{"missing table", ":8123", "/backend", true, adminsMissingTable, []bannerLine{listen, mounted,
|
||||
{"missing table", ":8123", "/backend", true, adminsMissingTable, "http://127.0.0.1:8123/backend", []bannerLine{
|
||||
{warn: true, text: "the admin tables do not exist yet; run summer migrate, then " + create}}},
|
||||
{"unknown", ":8123", "/backend", true, adminsUnknown, []bannerLine{listen, mounted,
|
||||
{"unknown", ":8123", "/backend", true, adminsUnknown, "http://127.0.0.1:8123/backend", []bannerLine{
|
||||
{text: "sign in with an administrator; " + create + " creates one"}}},
|
||||
{"off", ":8123", "", false, adminsUnknown, []bannerLine{listen,
|
||||
{text: "admin: off — no plugin registers an admin controller"},
|
||||
{text: "to add one, run summer make:admin-controller <vendor.plugin> <Name>, then summer build, summer migrate and " + create + "; the admin needs SUMMER_ADMIN__JWT__SECRET, which summer new writes to .env"}}},
|
||||
{"unparsable address", "8080", "/backend", true, adminsPresent, []bannerLine{
|
||||
{text: "listening on 8080"}, {text: "admin: mounted at /backend"}}},
|
||||
{"off", ":8123", "", false, adminsUnknown, "off — no plugin registers an admin controller", []bannerLine{
|
||||
{text: "to add an admin, run summer make:admin-controller <vendor.plugin> <Name>, then summer build, summer migrate and " + create + "; the admin needs SUMMER_ADMIN__JWT__SECRET, which summer new writes to .env"}}},
|
||||
{"unparsable address", "8080", "/backend", true, adminsPresent, "mounted at /backend", nil},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := serveBanner(tc.addr, tc.prefix, tc.mounted, tc.admins)
|
||||
if !slices.Equal(got, tc.want) {
|
||||
t.Fatalf("serveBanner:\n got %#v\nwant %#v", got, tc.want)
|
||||
value, notes := adminStatus(serveURL(tc.addr), tc.prefix, tc.mounted, tc.admins)
|
||||
if value != tc.value || !slices.Equal(notes, tc.notes) {
|
||||
t.Fatalf("adminStatus:\n got %q %#v\nwant %q %#v", value, notes, tc.value, tc.notes)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
382
modules/surf/summary.go
Normal file
382
modules/surf/summary.go
Normal file
@@ -0,0 +1,382 @@
|
||||
package surf
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/bonfire"
|
||||
"git.golem15.com/golem15/summercms/modules/conga"
|
||||
"git.golem15.com/golem15/summercms/modules/cooler"
|
||||
"git.golem15.com/golem15/summercms/modules/diary"
|
||||
"git.golem15.com/golem15/summercms/modules/lagoon"
|
||||
"git.golem15.com/golem15/summercms/modules/party"
|
||||
"github.com/jackc/pgx/v5/pgconn"
|
||||
)
|
||||
|
||||
// serveLog is the logger of serve's startup summary and shutdown report.
|
||||
var serveLog = diary.For("serve")
|
||||
|
||||
// Messages of the records serve logs.
|
||||
const (
|
||||
summaryMessage = "startup summary"
|
||||
reportMessage = "shutdown report"
|
||||
listenPrefix = "listening on "
|
||||
)
|
||||
|
||||
// requestCounter wraps the compiled handler and counts requests in flight
|
||||
// and served, whatever http.access_log says, for the shutdown report.
|
||||
type requestCounter struct {
|
||||
next http.Handler
|
||||
inFlight atomic.Int64
|
||||
served atomic.Int64
|
||||
}
|
||||
|
||||
func newRequestCounter(next http.Handler) *requestCounter {
|
||||
return &requestCounter{next: next}
|
||||
}
|
||||
|
||||
func (c *requestCounter) ServeHTTP(w http.ResponseWriter, req *http.Request) {
|
||||
c.inFlight.Add(1)
|
||||
defer func() {
|
||||
c.inFlight.Add(-1)
|
||||
c.served.Add(1)
|
||||
}()
|
||||
c.next.ServeHTTP(w, req)
|
||||
}
|
||||
|
||||
// pluginFact is one plugin's line of the verbose summary.
|
||||
type pluginFact struct {
|
||||
ID string
|
||||
Requires []string
|
||||
Pending int
|
||||
}
|
||||
|
||||
// summaryFacts is what serve reports once it is listening. It holds no DSN,
|
||||
// user, password, key or token: the database is its host, port and name.
|
||||
type summaryFacts struct {
|
||||
Header string
|
||||
Environment string
|
||||
Addr string
|
||||
URL string
|
||||
Admin string
|
||||
AdminNotes []bannerLine
|
||||
Database string
|
||||
// Pending is the number of pending migrations; -1 when it could not
|
||||
// be read.
|
||||
Pending int
|
||||
MigrateCmd string
|
||||
Plugins []pluginFact
|
||||
Queue string
|
||||
Scheduler string
|
||||
Mail string
|
||||
Cache string
|
||||
Realtime string
|
||||
Search string
|
||||
}
|
||||
|
||||
// factInputs are the values serve learned while booting.
|
||||
type factInputs struct {
|
||||
addr string
|
||||
bin string
|
||||
plugins []party.Plugin
|
||||
queues []string
|
||||
worker bool
|
||||
adminPrefix string
|
||||
mounted bool
|
||||
admins adminPresence
|
||||
pending []lagoon.MigrationRef
|
||||
pendingErr error
|
||||
version bonfire.Version
|
||||
}
|
||||
|
||||
// buildFacts collects the summary from the app's config and in.
|
||||
func buildFacts(app *backpack.App, in factInputs) summaryFacts {
|
||||
name := strings.TrimSpace(configString(app, "app.name"))
|
||||
if name == "" {
|
||||
name = strings.TrimPrefix(in.bin, "bin/")
|
||||
}
|
||||
f := summaryFacts{
|
||||
Header: in.version.Header(name),
|
||||
Environment: "unknown",
|
||||
Addr: in.addr,
|
||||
URL: serveURL(in.addr),
|
||||
Database: databaseFact(configString(app, "database.dsn")),
|
||||
Pending: len(in.pending),
|
||||
MigrateCmd: in.bin + " migrate",
|
||||
Mail: driverFact(app, "mail.driver", "memory"),
|
||||
Cache: driverFact(app, cooler.CacheDriverKey, "memory"),
|
||||
Realtime: driverFact(app, "realtime.driver", "null"),
|
||||
Search: driverFact(app, "search.driver", "null"),
|
||||
}
|
||||
if app != nil && app.Config != nil && app.Config.Environment() != "" {
|
||||
f.Environment = app.Config.Environment()
|
||||
}
|
||||
if in.pendingErr != nil {
|
||||
f.Pending = -1
|
||||
}
|
||||
f.Admin, f.AdminNotes = adminStatus(f.URL, in.adminPrefix, in.mounted, in.admins)
|
||||
perPlugin := map[string]int{}
|
||||
for _, ref := range in.pending {
|
||||
perPlugin[ref.Plugin]++
|
||||
}
|
||||
for _, p := range in.plugins {
|
||||
f.Plugins = append(f.Plugins, pluginFact{ID: p.ID(), Requires: p.Requires(), Pending: perPlugin[p.ID()]})
|
||||
}
|
||||
if in.worker {
|
||||
f.Queue = strings.Join(in.queues, ", ") + " (worker in serve)"
|
||||
f.Scheduler = "in serve"
|
||||
} else {
|
||||
f.Queue = "off in serve (queue.work_in_serve is false; run queue:work)"
|
||||
f.Scheduler = "off in serve (run schedule:run)"
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
func configString(app *backpack.App, key string) string {
|
||||
if app == nil || app.Config == nil {
|
||||
return ""
|
||||
}
|
||||
return app.Config.String(key)
|
||||
}
|
||||
|
||||
// driverFact is the lower-cased driver named by key, or def when empty.
|
||||
func driverFact(app *backpack.App, key, def string) string {
|
||||
if v := strings.ToLower(strings.TrimSpace(configString(app, key))); v != "" {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
// databaseFact is host:port/name parsed from dsn with pgconn.ParseConfig,
|
||||
// never the user or password. A DSN that does not parse is "configured",
|
||||
// an empty one "not configured".
|
||||
func databaseFact(dsn string) string {
|
||||
dsn = strings.TrimSpace(dsn)
|
||||
if dsn == "" {
|
||||
return "not configured"
|
||||
}
|
||||
cfg, err := pgconn.ParseConfig(dsn)
|
||||
if err != nil || cfg.Host == "" {
|
||||
return "configured"
|
||||
}
|
||||
host := cfg.Host
|
||||
if strings.HasPrefix(host, "/") {
|
||||
host = "socket " + host
|
||||
} else if strings.Contains(host, ":") {
|
||||
host = "[" + host + "]"
|
||||
}
|
||||
fact := host + ":" + strconv.Itoa(int(cfg.Port))
|
||||
if cfg.Database != "" {
|
||||
fact += "/" + cfg.Database
|
||||
}
|
||||
return fact
|
||||
}
|
||||
|
||||
// migrations is the migration line: "up to date", "N pending — run ..." or
|
||||
// "unknown".
|
||||
func (f summaryFacts) migrations() (string, bool) {
|
||||
switch {
|
||||
case f.Pending < 0:
|
||||
return "unknown (the migration history could not be read)", true
|
||||
case f.Pending == 0:
|
||||
return "up to date", false
|
||||
default:
|
||||
return fmt.Sprintf("%d pending — run %s", f.Pending, f.MigrateCmd), true
|
||||
}
|
||||
}
|
||||
|
||||
func (f summaryFacts) pluginIDs() []string {
|
||||
ids := make([]string, 0, len(f.Plugins))
|
||||
for _, p := range f.Plugins {
|
||||
ids = append(ids, p.ID)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
// details is the block printed on a terminal; verbosity 1 or more adds one
|
||||
// line per plugin with its required plugins and pending migrations.
|
||||
func (f summaryFacts) details(verbosity int) []bonfire.Detail {
|
||||
url := f.URL
|
||||
if url == "" {
|
||||
url = f.Addr
|
||||
}
|
||||
migrations, warn := f.migrations()
|
||||
plugins := strconv.Itoa(len(f.Plugins))
|
||||
if len(f.Plugins) > 0 {
|
||||
plugins += " (" + strings.Join(f.pluginIDs(), ", ") + ")"
|
||||
}
|
||||
rows := []bonfire.Detail{
|
||||
{Label: "Environment", Value: f.Environment},
|
||||
{Label: "URL", Value: url},
|
||||
{Label: "Admin", Value: f.Admin},
|
||||
{Label: "Database", Value: f.Database},
|
||||
{Label: "Migrations", Value: migrations, Warn: warn},
|
||||
{Label: "Plugins", Value: plugins},
|
||||
}
|
||||
if verbosity >= 1 {
|
||||
for _, p := range f.Plugins {
|
||||
requires := "requires nothing"
|
||||
if len(p.Requires) > 0 {
|
||||
requires = "requires " + strings.Join(p.Requires, ", ")
|
||||
}
|
||||
rows = append(rows, bonfire.Detail{
|
||||
Label: " " + p.ID,
|
||||
Value: fmt.Sprintf("%s; %d pending migrations", requires, p.Pending),
|
||||
Warn: p.Pending > 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
return append(rows,
|
||||
bonfire.Detail{Label: "Queue", Value: f.Queue},
|
||||
bonfire.Detail{Label: "Scheduler", Value: f.Scheduler},
|
||||
bonfire.Detail{Label: "Mail", Value: f.Mail},
|
||||
bonfire.Detail{Label: "Cache", Value: f.Cache},
|
||||
bonfire.Detail{Label: "Realtime", Value: f.Realtime},
|
||||
bonfire.Detail{Label: "Search", Value: f.Search},
|
||||
)
|
||||
}
|
||||
|
||||
// attrs are the attributes of the "startup summary" record.
|
||||
func (f summaryFacts) attrs() []slog.Attr {
|
||||
return []slog.Attr{
|
||||
slog.String("app", f.Header),
|
||||
slog.String("environment", f.Environment),
|
||||
slog.String("url", f.URL),
|
||||
slog.String("admin", f.Admin),
|
||||
slog.String("database", f.Database),
|
||||
slog.Int("migrations_pending", f.Pending),
|
||||
slog.Int("plugins", len(f.Plugins)),
|
||||
slog.String("plugin_ids", strings.Join(f.pluginIDs(), ",")),
|
||||
slog.String("queue", f.Queue),
|
||||
slog.String("scheduler", f.Scheduler),
|
||||
slog.String("mail", f.Mail),
|
||||
slog.String("cache", f.Cache),
|
||||
slog.String("realtime", f.Realtime),
|
||||
slog.String("search", f.Search),
|
||||
}
|
||||
}
|
||||
|
||||
// printSummary prints the block and the admin notes through out.
|
||||
func printSummary(out bonfire.Output, f summaryFacts) {
|
||||
bonfire.Details(out, f.Header, f.details(bonfire.VerbosityOf(out)))
|
||||
printBanner(out, f.AdminNotes)
|
||||
}
|
||||
|
||||
// logSummary logs one INFO record "startup summary" with every fact, one
|
||||
// whose message is exactly "listening on <addr>" (application smoke scripts
|
||||
// wait for it) and the admin notes (warnings at WARN).
|
||||
func logSummary(ctx context.Context, log *slog.Logger, f summaryFacts) {
|
||||
log.LogAttrs(ctx, slog.LevelInfo, summaryMessage, f.attrs()...)
|
||||
log.LogAttrs(ctx, slog.LevelInfo, listenPrefix+f.Addr, slog.String("url", f.URL))
|
||||
for _, n := range f.AdminNotes {
|
||||
level := slog.LevelInfo
|
||||
if n.warn {
|
||||
level = slog.LevelWarn
|
||||
}
|
||||
log.LogAttrs(ctx, level, n.text)
|
||||
}
|
||||
}
|
||||
|
||||
// shutdownReport is what serve did, printed and logged after a signal.
|
||||
type shutdownReport struct {
|
||||
// Draining is the number of requests in flight when the signal came.
|
||||
Draining int64
|
||||
// Aborted is the number still running at the drain deadline.
|
||||
Aborted int64
|
||||
Served int64
|
||||
Worker *conga.WorkerStats
|
||||
Uptime time.Duration
|
||||
Errors []error
|
||||
}
|
||||
|
||||
// Drained is the number of in-flight requests that finished in time.
|
||||
func (r shutdownReport) Drained() int64 {
|
||||
return max(0, r.Draining-r.Aborted)
|
||||
}
|
||||
|
||||
// Err joins the drain errors; nil means a clean stop (exit 0).
|
||||
func (r shutdownReport) Err() error {
|
||||
return errors.Join(r.Errors...)
|
||||
}
|
||||
|
||||
func (r shutdownReport) requestsLine() string {
|
||||
return fmt.Sprintf("%d served, %d drained, %d aborted", r.Served, r.Drained(), r.Aborted)
|
||||
}
|
||||
|
||||
func workerLine(s *conga.WorkerStats) string {
|
||||
if s == nil {
|
||||
return "not running in serve"
|
||||
}
|
||||
return fmt.Sprintf("%d done, %d failed, %d handed back to the queue", s.Done, s.Failed, s.HandedBack+s.InFlight)
|
||||
}
|
||||
|
||||
func errorsLine(errs []error) string {
|
||||
if len(errs) == 0 {
|
||||
return "none"
|
||||
}
|
||||
parts := make([]string, 0, len(errs))
|
||||
for _, e := range errs {
|
||||
parts = append(parts, e.Error())
|
||||
}
|
||||
return strings.Join(parts, "; ")
|
||||
}
|
||||
|
||||
// uptime rounds d for the report.
|
||||
func uptime(d time.Duration) string {
|
||||
switch {
|
||||
case d >= time.Minute:
|
||||
return d.Round(time.Second).String()
|
||||
case d >= time.Second:
|
||||
return d.Round(100 * time.Millisecond).String()
|
||||
default:
|
||||
return d.Round(time.Millisecond).String()
|
||||
}
|
||||
}
|
||||
|
||||
// details is the report as label/value lines.
|
||||
func (r shutdownReport) details() []bonfire.Detail {
|
||||
return []bonfire.Detail{
|
||||
{Label: "Requests", Value: r.requestsLine(), Warn: r.Aborted > 0},
|
||||
{Label: "Worker", Value: workerLine(r.Worker)},
|
||||
{Label: "Uptime", Value: uptime(r.Uptime)},
|
||||
{Label: "Errors", Value: errorsLine(r.Errors), Warn: len(r.Errors) > 0},
|
||||
}
|
||||
}
|
||||
|
||||
// attrs are the attributes of the "shutdown report" record.
|
||||
func (r shutdownReport) attrs() []slog.Attr {
|
||||
attrs := []slog.Attr{
|
||||
slog.Int64("served", r.Served),
|
||||
slog.Int64("drained", r.Drained()),
|
||||
slog.Int64("aborted", r.Aborted),
|
||||
slog.Int64("uptime_ms", r.Uptime.Milliseconds()),
|
||||
}
|
||||
if r.Worker != nil {
|
||||
attrs = append(attrs,
|
||||
slog.Int64("jobs_done", r.Worker.Done),
|
||||
slog.Int64("jobs_failed", r.Worker.Failed),
|
||||
slog.Int64("jobs_handed_back", r.Worker.HandedBack+r.Worker.InFlight),
|
||||
)
|
||||
}
|
||||
if len(r.Errors) > 0 {
|
||||
attrs = append(attrs, slog.String("errors", errorsLine(r.Errors)))
|
||||
}
|
||||
return attrs
|
||||
}
|
||||
|
||||
// logReport logs the report as one record: INFO, or ERROR with drain errors.
|
||||
func logReport(ctx context.Context, log *slog.Logger, r shutdownReport) {
|
||||
level := slog.LevelInfo
|
||||
if len(r.Errors) > 0 {
|
||||
level = slog.LevelError
|
||||
}
|
||||
log.LogAttrs(ctx, level, reportMessage, r.attrs()...)
|
||||
}
|
||||
Reference in New Issue
Block a user