diff --git a/README.md b/README.md index 8041c3f..0ecc2c8 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ It owns the `users` table extensions, `user_throttle`, `jwt_blacklist`, `user_ap ## Features - **Registration and activation.** `POST /register`, `POST /activate`, `POST /activate-by-code`. Registration can be switched off and throttled; `activate_mode` is `auto`, `user` (a mailed code) or `admin`, with codes from `IssueActivationCode` and `VerifyActivationCode`. A `/register` body sets only the `User.RegisterFillable` columns: `name`, `surname`, `email`, `password` (stored as its hash), `preferred_locale` and `marketing_consent`. It cannot set `organisation_id` or `organisation_role` (membership is the administrator's or the application's), `is_onboarded`, `must_change_password`, or the activation, reset, address proof, handle and permission columns; other keys are ignored (they still reach `RegisterEvent.Payload`). `created_ip_address` and `last_ip_address` are server-set from the client address only. With `require_activation` on (the default), `POST /login` refuses an account that is not activated with the same 401 invalid-login answer as a wrong password and counts the attempt for the login throttle. Such an account is activated by the mailed code (`POST /activate-by-code` answers with a token unless a login gate holds the sign-in back, for example a second-factor challenge the user completes through the gate's plugin; the account is activated either way), by the invitation link or by an administrator. `POST /activate` serves the sessions of not-activated accounts, which exist when `require_activation` is off. `Login` checks in this order: a ban or an active suspension (`RejectIfThrottled`), the password, the activation, then the deactivation restore. -- **JWT sessions.** `POST /login`, `GET /fetch`, `POST /refresh`, `POST /logout`. Boot registers the `jwt` guard with the framework's `bouncer.Registry` and exposes it as the `jwt.auth` middleware for other plugins. The guard reads the session cookies `classes.SessionCookieNames()` lists and resolves the published `bouncer.BlacklistStore` on every request. Boot installs the plugin's GORM hooks (`classes.RegisterHooks`), publishes the Postgres `jwt_blacklist` store (`bouncer.NewPostgresBlacklist`) and starts its background sweep, which prunes the blacklist, through `lagoon.OnDatabase`. Under `summer serve`, which publishes the database after Boot, they take effect when `lagoon.Publish` runs, before the router is built; when the database is already published they take effect at once. While no store is published, no token counts as blacklisted. `POST /logout` ends the session however the token travels: the bearer header, or only the `token` or `auth_token` cookie the guard also reads. Every session token the request carries goes to the Postgres `jwt_blacklist` until its refresh window closes (the later of `exp` and `iat` plus `refresh_ttl`, plus a minute), so the guard refuses it on the next request and `POST /refresh` cannot revive it after the access token expired; a token whose access lifetime passed is revoked too while it could still be refreshed. The answer, a 401 included, expires the `auth.cookie.*` token cookie (the one `golem15.sociallogin`'s `token_cookie` sets), and any other session cookie the request carries, with that cookie's path, domain, SameSite and Secure. `POST /refresh` loads the token's user and refuses (401, `Could not refresh token`, `User not found`) a deactivated or banned user and a token issued before the user's `tokens_valid_after`; a suspension alone does not stop a refresh. `GET /fetch`, `POST /refresh`, `POST /update` and `POST /change-password`, like the plugin's other signed-in routes, read the session token as the `jwt` guard does: the bearer header first, then the `token` and `auth_token` cookies, so a cookie-only session works on them and a bearer header wins when both are sent. A `POST /refresh` whose token came from the `auth.cookie.*` cookie (`auth.cookie.name`, default `token`) with no bearer header gets that cookie back with the rotated token: the cookie has the `bouncer.LoadCookieConfig` attributes, HttpOnly and Max-Age `refresh_ttl`, and the answer is `{}`, so the token never reaches JavaScript. This holds whoever wrote the cookie (this plugin's `golem15.user.jwt.token_cookie`, `golem15.sociallogin`'s `token_cookie` or the application); a frontend that writes the `token` cookie itself and refreshes through it gets it back HttpOnly, without a body token, so such a frontend sends the bearer header to keep rotating in the body. A refresh carried by the bearer header (a cookie sent with it included) or by the other session cookie answers `{"token": …}` and writes no cookie; the old token is blacklisted after `golem15.user.jwt.blacklist_grace`, so that caller stores the new token itself. A refused refresh writes and forgets no cookie. When `bouncer.LoadCookieConfig` refuses the `auth.cookie` block, a cookie-carried refresh answers the opaque 500 before anything is rotated; a bearer refresh never reads that block. Keep `auth.cookie.name` at `token` or `auth_token`, the cookies the guard reads. With `golem15.user.jwt.token_cookie` on, `POST /login`, `POST /register` (when it signs the new account in) and `POST /activate-by-code` set that HttpOnly session cookie (`bouncer.LoadCookieConfig` attributes, Max-Age `refresh_ttl`) and leave `token` out of the body; `user`, and `message` on activate-by-code, stay. With it off (the default) the token is in the body and no cookie is set. SameSite stays the `auth.cookie` setting (`lax` by default) and there is no CSRF token, so do not set `same_site: none` for this cookie. With the key on, a broken `auth.cookie` block answers those three routes with the opaque 500 before they sign in, create or activate anything. Personal API tokens (`POST /tokens`) are not sessions and are unaffected. A cookie-carried session relies on the cookie's SameSite attribute against cross-site requests (the `auth.cookie` default is `lax`), as on the `jwt.auth` routes. +- **JWT sessions.** `POST /login`, `GET /fetch`, `POST /refresh`, `POST /logout`. Boot registers the `jwt` guard with the framework's `bouncer.Registry` and exposes it as the `jwt.auth` middleware for other plugins. The guard reads the session cookies `classes.SessionCookieNames()` lists and resolves the published `bouncer.BlacklistStore` on every request. Boot installs the plugin's GORM hooks (`classes.RegisterHooks`) and publishes the Postgres `jwt_blacklist` store (`bouncer.NewPostgresBlacklist`) through `lagoon.OnDatabase`; the expired rows are deleted by the scheduled `user:jwt-blacklist-sweep` command (see Configuration). Under `summer serve`, which publishes the database after Boot, they take effect when `lagoon.Publish` runs, before the router is built; when the database is already published they take effect at once. While no store is published, no token counts as blacklisted. `POST /logout` ends the session however the token travels: the bearer header, or only the `token` or `auth_token` cookie the guard also reads. Every session token the request carries goes to the Postgres `jwt_blacklist` until its refresh window closes (the later of `exp` and `iat` plus `refresh_ttl`, plus a minute), so the guard refuses it on the next request and `POST /refresh` cannot revive it after the access token expired; a token whose access lifetime passed is revoked too while it could still be refreshed. The answer, a 401 included, expires the `auth.cookie.*` token cookie (the one `golem15.sociallogin`'s `token_cookie` sets), and any other session cookie the request carries, with that cookie's path, domain, SameSite and Secure. `POST /refresh` loads the token's user and refuses (401, `Could not refresh token`, `User not found`) a deactivated or banned user and a token issued before the user's `tokens_valid_after`; a suspension alone does not stop a refresh. `GET /fetch`, `POST /refresh`, `POST /update` and `POST /change-password`, like the plugin's other signed-in routes, read the session token as the `jwt` guard does: the bearer header first, then the `token` and `auth_token` cookies, so a cookie-only session works on them and a bearer header wins when both are sent. A `POST /refresh` whose token came from the `auth.cookie.*` cookie (`auth.cookie.name`, default `token`) with no bearer header gets that cookie back with the rotated token: the cookie has the `bouncer.LoadCookieConfig` attributes, HttpOnly and Max-Age `refresh_ttl`, and the answer is `{}`, so the token never reaches JavaScript. This holds whoever wrote the cookie (this plugin's `golem15.user.jwt.token_cookie`, `golem15.sociallogin`'s `token_cookie` or the application); a frontend that writes the `token` cookie itself and refreshes through it gets it back HttpOnly, without a body token, so such a frontend sends the bearer header to keep rotating in the body. A refresh carried by the bearer header (a cookie sent with it included) or by the other session cookie answers `{"token": …}` and writes no cookie; the old token is blacklisted after `golem15.user.jwt.blacklist_grace`, so that caller stores the new token itself. A refused refresh writes and forgets no cookie. When `bouncer.LoadCookieConfig` refuses the `auth.cookie` block, a cookie-carried refresh answers the opaque 500 before anything is rotated; a bearer refresh never reads that block. Keep `auth.cookie.name` at `token` or `auth_token`, the cookies the guard reads. With `golem15.user.jwt.token_cookie` on, `POST /login`, `POST /register` (when it signs the new account in) and `POST /activate-by-code` set that HttpOnly session cookie (`bouncer.LoadCookieConfig` attributes, Max-Age `refresh_ttl`) and leave `token` out of the body; `user`, and `message` on activate-by-code, stay. With it off (the default) the token is in the body and no cookie is set. SameSite stays the `auth.cookie` setting (`lax` by default) and there is no CSRF token, so do not set `same_site: none` for this cookie. With the key on, a broken `auth.cookie` block answers those three routes with the opaque 500 before they sign in, create or activate anything. Personal API tokens (`POST /tokens`) are not sessions and are unaffected. A cookie-carried session relies on the cookie's SameSite attribute against cross-site requests (the `auth.cookie` default is `lax`), as on the `jwt.auth` routes. - **Login gates.** Register publishes a `*classes.LoginGates` registry. Another plugin (a two-factor plugin, for example) looks it up with `app.Lookup[*classes.LoginGates]()` and adds a `classes.LoginGate` in its Boot. `POST /login` consults the gates through `controllers.CheckLoginGates` after the password, activation and throttle checks (and the bcrypt rehash) and before it restores a deactivated account, sends the reactivate mail, stamps `last_seen` and mints the token. `POST /activate-by-code` consults them after the mailed code verified and before it mints the token, with source `password`: with `require_activation` off an account can enable a second factor before it activates, so the mailed code must not stand in for that factor. A gate sees a `classes.LoginAttempt` (`User`, `Source`, the already-read login or activation body as `Fields`, `Request`). A nil result allows the sign-in; a `classes.LoginGateResult` replaces the token response with its `Status` (0 means 200) and `Body`, for example `{"two_factor_required":true,"challenge_token":…,"available_methods":[…],"expires_at":…}`, and no token is issued, no session cookie is set, no account is restored and no login event fires. A gate error fails closed with the opaque 500. An activation a gate held back or failed stays committed. Without a gate, `/login` and `/activate-by-code` answer exactly as before. `POST /refresh` never consults the gates. - **Login event.** Every sign-in that issued a session token fires `classes.LoginEvent` (`classes.LoginEventName`, `golem15.user.login`, PHP's name) with `User` and `Source` (`LoginSourcePassword`, `LoginSourceSocial`, `LoginSourcePasskey`, `LoginSourceTwoFactor`, `LoginSourceDevice`), after the token was minted. `POST /login` fires it with `password`; another plugin that completes a sign-in fires it through `controllers.FireLoginEvent`. A listener error is logged and never fails the sign-in. Refresh and `/register` fire none. - **Password-change hook.** A password reset, the user's own `POST /change-password` and an administrator setting a password on the Users screen fire `classes.AccessRevokedEvent` inside the write's transaction (see Access revocation); `(*AccessRevokedEvent).PasswordChanged()` reports these three reasons. A two-factor plugin listens to it to revoke trusted devices, and this plugin revokes the users' pending and confirmed device sessions (see Device authorization); a listener error rolls the password write back. The bcrypt rehash on login and a new account fire nothing. @@ -251,6 +251,7 @@ Packages: - `OrganisationLogoURL(ctx, app, org, size)`, `OrganisationLogoFields(ctx, app, org)` and `OrganisationLogoSize`, the organisation logo for an application's organisation payloads (see Organisations). - `AvatarURL(ctx, app, user, size)`, the display picture rule of the user payload at any size: the avatar thumbnail (size by size, mode auto) when the user has an avatar and a file bucket is published, the original file's public URL (`attach.File.URL`, which follows `storage.uploads.public_path_prefix` and `public_url`) when the thumbnail fails, otherwise the gravatar URL. The URL may be protocol-relative or relative. The payload's `avatar_url` comes from it at 128. - `BackendLinks` (`CreateLinkRequest`, `LinkRequestByCode`, `ConfirmLinkRequest`, `SetLink`, `RemoveLink`, `LinkFor`, `LinkOf`, `LinkedBackendUser`, `RegisterBackendLinkListeners`), its errors `ErrLinkRequestNotFound`, `ErrLinkRequestUsed`, `ErrLinkRequestExpired`, `ErrStaleSession`, `ErrAlreadyLinked`, `ErrBackendUserUnavailable` and `ErrFrontendUserUnavailable`, the events `BackendLinkedEvent` / `BackendLinkedEventName` and `BackendUnlinkedEvent` / `BackendUnlinkedEventName`, `BackendLinkRequestTTL(app)`, `DefaultBackendLinkRequestTTL`, `FrontendURL(app)`, `BackendLinkPagePath(app)`, `DefaultBackendLinkPagePath`, `BackendLinksFor(app)` (the published service, or one over the app) and `BackendDisplayName(admin)`. + - `BlacklistSweepInterval(app)` reads `golem15.user.jwt.blacklist_sweep_interval` (`DefaultBlacklistSweepInterval` when unset, 0 for no sweep, an error for an interval the scheduler cannot run); `JWTBlacklistTable` is the blacklist store's table. - `PermissionManageBackendLinks`, `FrontendPermissionCatalog(ctx, app, db)`, `MCPUserGuard`, `FireSubjectChanged(ctx, app, guard, userID, kind)` and `FireSubjectChangedAfterCommit(ctx, app, tx, guard, userID, kind)` with the guard names `SubjectGuardUser` and `SubjectGuardBackend`, and `FireSubjectEvent(ctx, app, ev)` and `FireSubjectEventAfterCommit(ctx, app, tx, ev)`, which fire a whole `pact.MCPSubjectChanged` (with `TokenID` or `ClientID` to end only one token's or one client's sessions). - `AccessRevokedEvent` / `AccessRevokedEventName` and `FireAccessRevoked(ctx, app, tx, userIDs, before, reason)` (see Access revocation). - `LoginAttempt`, `LoginGateResult`, `LoginGate` (`CheckLogin(ctx, attempt)`), `LoginGates` with `NewLoginGates()`, `Add(gate)` and `Check(ctx, attempt)`, the login-gate seam (see Login gates), and the sources `LoginSourcePassword`, `LoginSourceSocial`, `LoginSourcePasskey`, `LoginSourceTwoFactor` and `LoginSourceDevice`. @@ -262,7 +263,7 @@ Packages: - `middleware`: `APITokenGuard`, `Scope`, `Permission` and `BearerToken`. - `models`: `User`, `Throttle`, `Organisation`, `UserGroup`, `UsersGroup` (the `users_groups` pivot), `APIToken`, `OAuthIdentity`, `FrontendPermission`, `BackendLink`, `BackendLinkRequest`, `DeviceAuthSession` (with `DeviceStatusPending`, `DeviceStatusConfirmed`, `DeviceStatusUsed`, `DeviceStatusExpired` and `DeviceStatusRevoked`) and `PermissionSet`. `PermissionSet` reads a stored permission column tolerantly (NULL, an empty string, `[]`, numeric strings) and writes a JSON object of integers; `ParsePermissionSet` and `PermissionSet.Text` do the same for a column that is a plain string on its model. `UserGroup.UsersCount` is filled only by a query that selects `users_count` (the admin list does). `Organisation.Members` is the relation behind the members manager, `Organisation.AttachRelations` declares its avatar (`OrganisationAvatarField`) and its logo (`OrganisationLogoField`), and `Slugify` is the slug rule. `User.FilterScope` answers the admin list scope `FilterByGroup`, and `User.AttachRelations` declares the avatar (`AvatarField`). `User.RegisterFillable` is the registration allow-list; `User.Fillable` is the wider mass-assignment backstop of the admin Users form (cabana intersects it with the form's fields) and of trusted application code. - `updates`: the gormigrate migrations, returned by `updates.All()`. -- `console`: `RequirePasswordChange`, `BackfillHandles` and `APITokenCommands`. +- `console`: `RequirePasswordChange`, `BackfillHandles`, `JWTBlacklistSweep` and `APITokenCommands`. ## Configuration @@ -275,7 +276,7 @@ Defaults live in `config/config.yaml` and are merged under `golem15.user.*`. Any | `golem15.user.jwt.refresh_ttl` | `20160` | Refresh window in minutes. | | `golem15.user.jwt.blacklist_grace` | `0` | Grace period in seconds for a blacklisted token. | | `golem15.user.jwt.leeway` | `0` | Clock leeway in seconds (declared for parity, not read yet). | -| `golem15.user.jwt.blacklist_sweep_interval` | `10m` | How often expired blacklist rows are deleted (a Go duration). | +| `golem15.user.jwt.blacklist_sweep_interval` | `10m` | How often the scheduler runs `user:jwt-blacklist-sweep`, which deletes the `jwt_blacklist` rows whose `expires_at` has passed (a Go duration). It must be at least 1s and divide 24h evenly (`1m`, `10m`, `1h`); `0` schedules no sweep. A value that does not parse, is negative, is shorter than a second or does not divide 24h fails boot. The sweep runs on the job worker's scheduled queue: the in-process worker of `summer serve` while `queue.work_in_serve` is on, `queue:work`, or `schedule:run`; with no worker on that queue, run `schedule:run --once` from cron or the command by hand. The schedule runs once per period across instances. | | `golem15.user.jwt.token_cookie` | `false` | When on, `POST /login`, `POST /register` (when it signs the new account in) and `POST /activate-by-code` set the HttpOnly session cookie with the `auth.cookie.*` attributes (`bouncer.LoadCookieConfig`; Max-Age `refresh_ttl`) and leave `token` out of the body (`user`, and `message` on activate-by-code, stay). When off, the token is in the body and no cookie is set. The twin of `golem15.sociallogin.token_cookie`. | | `golem15.user.activation.require_activation` | `true` | Whether an account must be activated before login. When `true`, `POST /login` refuses a not-activated account (the opaque invalid-login answer, counted as a failed attempt) and `/register` in `admin` mode returns no token; when `false`, both sign such an account in. | | `golem15.user.activation.activate_mode` | `auto` | `auto`, `user` (mailed code) or `admin`. | @@ -305,6 +306,7 @@ Defaults live in `config/config.yaml` and are merged under `golem15.user.*`. Any |---------|---------|---------| | `user:require-password-change ` | Flag an account so its next session must set a new password. Ends with `Password change required for `. | `bin/ user:require-password-change jane@example.com` | | `user:backfill-handles` | Assign a unique handle to every user row that still has none, with a progress bar. Ends with `Assigned handles to N user(s)`. | `bin/ user:backfill-handles` | +| `user:jwt-blacklist-sweep` | Delete the `jwt_blacklist` rows whose `expires_at` has passed; the scheduler runs it every `golem15.user.jwt.blacklist_sweep_interval`. Ends with `Deleted the expired JWT blacklist rows.` | `bin/ user:jwt-blacklist-sweep` | | `user:token:create [--name api] [--scopes read,write] [--expires RFC3339] [--json]` | Create an application-wide API token and print its secret once (also under `-q`); `--json` prints the token and its details as one object. | `bin/ user:token:create jane@example.com --scopes read --json` | | `user:token:list` | List token metadata without secrets or hashes; `--json` prints a JSON array with the expiry, revocation and last-use times. | `bin/ user:token:list --json` | | `user:token:revoke ` | Revoke a shared API token immediately. | `bin/ user:token:revoke 12` | @@ -337,6 +339,8 @@ The mail language is pinned by `TestNormalizeMailLocale`, `TestResolveMailLocale `TestLogoutEndsTheSession` signs in through `POST /login` and logs out with the bearer header, with only the `token` or `auth_token` cookie, and with both a bearer and a different cookie token, then checks that the token is refused on a `jwt.auth` route, at `/fetch` and at `/refresh`, each through every carrier (bearer header, `token` cookie, `auth_token` cookie), that the forget cookies match the token cookie's attributes (also with a configured domain and SameSite), that the blacklist row lasts for the refresh window so `/refresh` stays refused after the sweep that follows the access token's expiry, that an expired but refreshable token is revoked, and which tokens a logout refuses with 401. +The blacklist sweep is pinned by `TestBlacklistSweepInterval` (in `classes`, `-short`: the default, valid intervals, `0` as a string, a duration, an int and a `SUMMER_` variable, and the values that must fail), `TestBlacklistSweepSchedule` (root, `-short`: the default and custom `Schedule()` entries, none for `0` or a bad value, every scheduled command registered), `TestBlacklistSweepBoot` (root, `-short`: Boot fails naming the key for each bad value and accepts `10m`, `0` and an unset key), `TestJWTBlacklistSweepCommand` (root, `-short`: one `Sweep` through the published store with the result line, a failing store, a nil app) and `TestJWTBlacklistSweepPostgres` (Postgres, skipped under `-short`: the command deletes the expired row and keeps the valid one, through the published store and through a database opened from `database.dsn`). + `TestBootBeforeDatabase` boots in the serve order (`party.Activate` with no database, then `lagoon.Publish`, through `newServeOrderEnv` in `admin_harness_test.go`). It checks that the blacklist store and the GORM hooks are installed on Publish and not before, and that a token ended by `POST /logout` is refused on a `jwt.auth` route through every carrier, at `/fetch` and at `/refresh`, with its `jwt_blacklist` row. `TestAppBlacklist` (in `-short`) covers the guard's per-request store lookup: no store, delegation and a failing store. `TestSessionRoutesAcceptCookies` checks that a session `jwt.auth` accepts through only the `token` or `auth_token` cookie is served by `GET /fetch`, `POST /refresh` (through `token`, the `auth.cookie` default, the rotated token comes back in a rewritten cookie with a `{}` body; through `auth_token` it comes in the body with no cookie; the old token is refused afterwards), `POST /update`, `POST /change-password` (the calling token stays valid) and `POST /marketing-consent`; that a request without credentials still gets the old 401 bodies; that the bearer header wins over a cookie exactly as on `jwt.auth`, an invalid bearer included; and that a cut-off token and a banned user are refused through the cookie. diff --git a/blacklist_sweep_test.go b/blacklist_sweep_test.go new file mode 100644 index 0000000..315c0a1 --- /dev/null +++ b/blacklist_sweep_test.go @@ -0,0 +1,283 @@ +package user + +import ( + "bytes" + "context" + "encoding/base64" + "errors" + "os" + "path/filepath" + "strings" + "sync" + "testing" + "time" + + "git.golem15.com/golem15/summercms/modules/backpack" + "git.golem15.com/golem15/summercms/modules/bonfire" + "git.golem15.com/golem15/summercms/modules/bouncer" + "git.golem15.com/golem15/summercms/modules/compass" + "git.golem15.com/golem15/summercms/modules/pact" + "gorm.io/gorm" +) + +const ( + sweepCommand = "user:jwt-blacklist-sweep" + sweepIntervalKey = "golem15.user.jwt.blacklist_sweep_interval" +) + +// badSweepIntervals are values the scheduler cannot run, so Boot rejects them. +var badSweepIntervals = []string{"soon", "3", "-1m", "500ms", "7m", "48h"} + +// fakeBlacklist is a bouncer.BlacklistStore that records the Sweep calls. +type fakeBlacklist struct { + mu sync.Mutex + sweeps []time.Time + sweepErr error +} + +func (f *fakeBlacklist) Add(context.Context, string, time.Time, time.Time) error { return nil } + +func (f *fakeBlacklist) IsBlacklisted(context.Context, string) (bool, error) { + return false, nil +} + +func (f *fakeBlacklist) Sweep(_ context.Context, now time.Time) error { + f.mu.Lock() + defer f.mu.Unlock() + f.sweeps = append(f.sweeps, now) + return f.sweepErr +} + +func (f *fakeBlacklist) calls() []time.Time { + f.mu.Lock() + defer f.mu.Unlock() + return append([]time.Time(nil), f.sweeps...) +} + +// TestBlacklistSweepSchedule schedules user:jwt-blacklist-sweep every 10m +// by default, before Register (nil app) too, at a custom interval, not at +// all for 0 or a bad value, and every scheduled command is a registered +// command. +func TestBlacklistSweepSchedule(t *testing.T) { + want := pact.ScheduledCommand{Command: sweepCommand, Cadence: pact.Every(10 * time.Minute)} + app := bareApp(t) + for name, p := range map[string]*Plugin{"unset": {app: app}, "nil app": {}} { + s := p.Schedule() + if len(s) != 1 || s[0].Command != want.Command || s[0].Cadence != want.Cadence || len(s[0].Args) != 0 { + t.Fatalf("%s: schedule = %+v", name, s) + } + } + names := map[string]bool{} + for _, c := range (&Plugin{app: app}).Commands() { + names[c.Name] = true + } + for _, s := range (&Plugin{app: app}).Schedule() { + if !names[s.Command] { + t.Fatalf("scheduled %s is not registered: %v", s.Command, names) + } + } + + for value, every := range map[string]time.Duration{"1h": time.Hour, "30s": 30 * time.Second} { + app := bareApp(t) + if err := app.Config.Set(sweepIntervalKey, value); err != nil { + t.Fatal(err) + } + s := (&Plugin{app: app}).Schedule() + if len(s) != 1 || s[0].Command != sweepCommand || s[0].Cadence != pact.Every(every) { + t.Fatalf("%s: schedule = %+v", value, s) + } + } + for _, off := range []string{"0", "0s"} { + app := bareApp(t) + if err := app.Config.Set(sweepIntervalKey, off); err != nil { + t.Fatal(err) + } + if s := (&Plugin{app: app}).Schedule(); len(s) != 0 { + t.Fatalf("%s: schedule = %+v, want none", off, s) + } + } + if s := (&Plugin{app: envApp(t, "SUMMER_GOLEM15__USER__JWT__BLACKLIST_SWEEP_INTERVAL=0")}).Schedule(); len(s) != 0 { + t.Fatalf("SUMMER_ 0: schedule = %+v, want none", s) + } + for _, bad := range badSweepIntervals { + app := bareApp(t) + if err := app.Config.Set(sweepIntervalKey, bad); err != nil { + t.Fatal(err) + } + if s := (&Plugin{app: app}).Schedule(); s != nil { + t.Fatalf("%s: schedule = %+v, want nil", bad, s) + } + } +} + +// envApp is a bare application whose config also reads environ. +func envApp(t *testing.T, environ ...string) *backpack.App { + t.Helper() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "app.yaml"), []byte("name: user-env\n"), 0o644); err != nil { + t.Fatal(err) + } + cfg, err := compass.Open(compass.Options{Dir: dir, Environ: append([]string{"SUMMER_ENV=development"}, environ...)}) + if err != nil { + t.Fatal(err) + } + return backpack.New(cfg) +} + +// TestBlacklistSweepBoot fails Boot on an interval the scheduler cannot run, +// naming the key, and accepts 10m, 0 and an unset key. +func TestBlacklistSweepBoot(t *testing.T) { + withSecret := func(t *testing.T) *backpack.App { + app := bareApp(t) + if err := app.Config.Set("golem15.user.jwt.secret", "blacklist-sweep-boot-secret"); err != nil { + t.Fatal(err) + } + return app + } + for _, bad := range badSweepIntervals { + app := withSecret(t) + if err := app.Config.Set(sweepIntervalKey, bad); err != nil { + t.Fatal(err) + } + if err := (&Plugin{}).Boot(app); err == nil || !strings.Contains(err.Error(), sweepIntervalKey) { + t.Fatalf("Boot with %q: %v", bad, err) + } + } + for _, good := range []string{"10m", "0", ""} { + app := withSecret(t) + if good != "" { + if err := app.Config.Set(sweepIntervalKey, good); err != nil { + t.Fatal(err) + } + } + if err := (&Plugin{}).Boot(app); err != nil && strings.Contains(err.Error(), sweepIntervalKey) { + t.Fatalf("Boot with %q: %v", good, err) + } + } +} + +// TestJWTBlacklistSweepCommand runs the command over a published store: one +// Sweep with the current time and the result line; a failing store is an +// error and no result line; a nil app is an error. +func TestJWTBlacklistSweepCommand(t *testing.T) { + run := func(t *testing.T, p *Plugin) (string, error) { + t.Helper() + cmd := commandNamed(t, p.Commands(), sweepCommand) + var buf bytes.Buffer + err := cmd.Run(context.Background(), emailArg{}, bonfire.NewOutput(nil, &buf, &buf)) + return buf.String(), err + } + + t.Run("published store", func(t *testing.T) { + app := bareApp(t) + fake := &fakeBlacklist{} + if err := app.Publish[bouncer.BlacklistStore](fake); err != nil { + t.Fatal(err) + } + before := time.Now() + out, err := run(t, &Plugin{app: app}) + after := time.Now() + if err != nil { + t.Fatalf("run: %v (output %q)", err, out) + } + calls := fake.calls() + if len(calls) != 1 || calls[0].Before(before) || calls[0].After(after) { + t.Fatalf("sweeps = %v, want one between %v and %v", calls, before, after) + } + if !strings.Contains(out, "Deleted the expired JWT blacklist rows.") { + t.Fatalf("output = %q", out) + } + }) + + t.Run("failing store", func(t *testing.T) { + app := bareApp(t) + down := errors.New("blacklist down") + if err := app.Publish[bouncer.BlacklistStore](&fakeBlacklist{sweepErr: down}); err != nil { + t.Fatal(err) + } + out, err := run(t, &Plugin{app: app}) + if !errors.Is(err, down) { + t.Fatalf("run error = %v, want it to wrap %v", err, down) + } + if strings.Contains(out, "Deleted") { + t.Fatalf("output after a failure = %q", out) + } + }) + + t.Run("nil app", func(t *testing.T) { + if _, err := run(t, &Plugin{}); err == nil { + t.Fatal("run with a nil app: no error") + } + }) +} + +// TestJWTBlacklistSweepPostgres runs user:jwt-blacklist-sweep against +// Postgres: through the store Boot published it deletes the expired row and +// keeps the one still valid; as a console command with no published +// database it opens one from database.dsn. +func TestJWTBlacklistSweepPostgres(t *testing.T) { + skipShortPostgres(t) + gdb, sqlDB := sessionDB(t) + insert := func(t *testing.T, jti string, at time.Time) { + t.Helper() + if err := gdb.Exec(`INSERT INTO jwt_blacklist (jti, expires_at, valid_until) VALUES (?, ?, ?)`, jti, at, at).Error; err != nil { + t.Fatal(err) + } + t.Cleanup(func() { gdb.Exec(`DELETE FROM jwt_blacklist WHERE jti = ?`, jti) }) + } + count := func(t *testing.T, jti string) int64 { + t.Helper() + var n int64 + if err := gdb.Raw(`SELECT COUNT(*) FROM jwt_blacklist WHERE jti = ?`, jti).Scan(&n).Error; err != nil { + t.Fatal(err) + } + return n + } + run := func(t *testing.T, p *Plugin) string { + t.Helper() + var buf bytes.Buffer + if err := commandNamed(t, p.Commands(), sweepCommand).Run(context.Background(), emailArg{}, bonfire.NewOutput(nil, &buf, &buf)); err != nil { + t.Fatalf("run: %v (output %q)", err, buf.String()) + } + return buf.String() + } + + t.Run("published store", func(t *testing.T) { + p := bootPlugin(t, newSessionApp(t, gdb, sqlDB), "0s") + insert(t, "sweep-me-published", time.Now().Add(-time.Minute)) + insert(t, "sweep-keep-published", time.Now().Add(time.Hour)) + out := run(t, p) + if count(t, "sweep-me-published") != 0 { + t.Fatal("the expired row was not deleted") + } + if count(t, "sweep-keep-published") != 1 { + t.Fatal("the unexpired row was deleted") + } + if !strings.Contains(out, "Deleted the expired JWT blacklist rows.") { + t.Fatalf("output = %q", out) + } + }) + + t.Run("database from config", func(t *testing.T) { + app := backpack.New(sessionConfig(t)) + if err := app.Config.Set("database.dsn", sessionDSN); err != nil { + t.Fatal(err) + } + // Opening a database also loads the encryption keys from app.key. + if err := app.Config.Set("app.key", base64.StdEncoding.EncodeToString([]byte("0123456789abcdef0123456789abcdef"))); err != nil { + t.Fatal(err) + } + if _, ok := app.Lookup[*gorm.DB](); ok { + t.Fatal("setup: a database is published") + } + insert(t, "sweep-me-console", time.Now().Add(-time.Minute)) + insert(t, "sweep-keep-console", time.Now().Add(time.Hour)) + run(t, &Plugin{app: app}) + if count(t, "sweep-me-console") != 0 { + t.Fatal("the expired row was not deleted") + } + if count(t, "sweep-keep-console") != 1 { + t.Fatal("the unexpired row was deleted") + } + }) +} diff --git a/classes/blacklist_sweep.go b/classes/blacklist_sweep.go new file mode 100644 index 0000000..a986ace --- /dev/null +++ b/classes/blacklist_sweep.go @@ -0,0 +1,52 @@ +package classes + +import ( + "fmt" + "strings" + "time" + + "git.golem15.com/golem15/summercms/modules/backpack" +) + +// JWTBlacklistTable is the table of migration +// 202609220007_create_jwt_blacklist that the plugin's blacklist store uses. +const JWTBlacklistTable = "jwt_blacklist" + +// DefaultBlacklistSweepInterval is how often the scheduler runs +// user:jwt-blacklist-sweep when golem15.user.jwt.blacklist_sweep_interval is +// not set. +const DefaultBlacklistSweepInterval = 10 * time.Minute + +const blacklistSweepIntervalKey = "golem15.user.jwt.blacklist_sweep_interval" + +// BlacklistSweepInterval reads golem15.user.jwt.blacklist_sweep_interval, a +// Go duration (10m by default). 0 means no sweep is scheduled. Any other +// value must be at least 1s and divide 24h evenly: the scheduler's Every +// cadence is aligned to local midnight, so a period that does not divide the +// day cannot be scheduled. A value that does not parse, is negative, is +// shorter than a second or does not divide 24h is an error, so a typo fails +// the boot instead of silently stopping the sweep. +func BlacklistSweepInterval(app *backpack.App) (time.Duration, error) { + if app == nil || app.Config == nil { + return DefaultBlacklistSweepInterval, nil + } + raw := strings.TrimSpace(app.Config.String(blacklistSweepIntervalKey)) + if raw == "" { + // A non-string value (an unquoted YAML 0) may read as "". + if v, ok := app.Config.Lookup(blacklistSweepIntervalKey); ok && v != nil { + raw = strings.TrimSpace(fmt.Sprint(v)) + } + } + if raw == "" { + return DefaultBlacklistSweepInterval, nil + } + d, err := time.ParseDuration(raw) + switch { + case err != nil: + case d == 0: + return 0, nil + case d >= time.Second && (24*time.Hour)%d == 0: + return d, nil + } + return 0, fmt.Errorf("golem15.user: %s must be 0 (no sweep) or a duration of at least 1s that divides 24h evenly, such as 10m, got %q", blacklistSweepIntervalKey, raw) +} diff --git a/classes/blacklist_sweep_test.go b/classes/blacklist_sweep_test.go new file mode 100644 index 0000000..13ef39a --- /dev/null +++ b/classes/blacklist_sweep_test.go @@ -0,0 +1,59 @@ +package classes + +import ( + "strings" + "testing" + "time" + + "git.golem15.com/golem15/summercms/modules/backpack" +) + +// TestBlacklistSweepInterval reads golem15.user.jwt.blacklist_sweep_interval: +// 10m by default, 0 for no sweep (string, duration or unquoted YAML int, and +// from a SUMMER_ variable), and an error naming the key for a value the +// scheduler cannot run. +func TestBlacklistSweepInterval(t *testing.T) { + const key = "golem15.user.jwt.blacklist_sweep_interval" + if got, err := BlacklistSweepInterval(nil); got != DefaultBlacklistSweepInterval || err != nil { + t.Errorf("nil app: %v %v", got, err) + } + if got, err := BlacklistSweepInterval(&backpack.App{}); got != DefaultBlacklistSweepInterval || err != nil { + t.Errorf("nil config: %v %v", got, err) + } + if got, err := BlacklistSweepInterval(mailTestApp(t)); got != 10*time.Minute || err != nil { + t.Errorf("unset: %v %v", got, err) + } + for _, tc := range []struct { + value any + want time.Duration + }{ + {"10m", 10 * time.Minute}, + {"1h", time.Hour}, + {"30s", 30 * time.Second}, + {"24h", 24 * time.Hour}, + {" 1m ", time.Minute}, + {"0", 0}, + {"0s", 0}, + {0, 0}, + } { + app := mailTestApp(t) + if err := app.Config.Set(key, tc.value); err != nil { + t.Fatal(err) + } + if got, err := BlacklistSweepInterval(app); got != tc.want || err != nil { + t.Errorf("%#v: %v %v, want %v", tc.value, got, err, tc.want) + } + } + if got, err := BlacklistSweepInterval(mailTestApp(t, "SUMMER_GOLEM15__USER__JWT__BLACKLIST_SWEEP_INTERVAL=0")); got != 0 || err != nil { + t.Errorf("SUMMER_ 0: %v %v", got, err) + } + for _, bad := range []any{"soon", "3", "-1m", "500ms", "7m", "48h", -5} { + app := mailTestApp(t) + if err := app.Config.Set(key, bad); err != nil { + t.Fatal(err) + } + if got, err := BlacklistSweepInterval(app); err == nil || got != 0 || !strings.Contains(err.Error(), key) { + t.Errorf("%#v: %v %v, want an error naming %s", bad, got, err, key) + } + } +} diff --git a/config/config.yaml b/config/config.yaml index 0b319bb..365aad6 100644 --- a/config/config.yaml +++ b/config/config.yaml @@ -4,6 +4,7 @@ jwt: refresh_ttl: 20160 blacklist_grace: 0 leeway: 0 + # 0 schedules no sweep; any other value must divide 24h (1m, 10m, 1h). blacklist_sweep_interval: 10m # On: POST /login, /register (when it signs the account in) and # /activate-by-code set the HttpOnly session cookie from auth.cookie.* and diff --git a/console/jwt_blacklist_sweep.go b/console/jwt_blacklist_sweep.go new file mode 100644 index 0000000..05ba413 --- /dev/null +++ b/console/jwt_blacklist_sweep.go @@ -0,0 +1,55 @@ +package console + +import ( + "context" + "fmt" + "time" + + "git.golem15.com/golem15/sm-user-plugin/classes" + "git.golem15.com/golem15/summercms/modules/backpack" + "git.golem15.com/golem15/summercms/modules/bonfire" + "git.golem15.com/golem15/summercms/modules/bouncer" + "git.golem15.com/golem15/summercms/modules/lagoon" +) + +// JWTBlacklistSweep deletes the expired rows of the JWT blacklist. It uses +// the blacklist store the plugin published or, when it runs as a console +// command without a published database, a store over a database opened from +// the config for the duration of the run. +func JWTBlacklistSweep(app *backpack.App) bonfire.Command { + return bonfire.Command{ + Name: "user:jwt-blacklist-sweep", + Description: "Delete the expired JWT blacklist rows", + Long: `Deletes the jwt_blacklist rows whose expires_at has passed. A row that +still blocks a token or its refresh is not touched, so running it twice +changes nothing. + +The scheduler runs it every golem15.user.jwt.blacklist_sweep_interval on +the job worker's scheduled queue. Run it by hand or from cron when no +worker runs that queue. It ends with "Deleted the expired JWT blacklist +rows."`, + Examples: []bonfire.Example{ + {Command: "user:jwt-blacklist-sweep", Description: "Delete the expired JWT blacklist rows"}, + {Command: "user:jwt-blacklist-sweep -q", Description: "Run it quietly, printing only the result line"}, + }, + Run: func(ctx context.Context, _ bonfire.Input, out bonfire.Output) error { + if app == nil { + return fmt.Errorf("user:jwt-blacklist-sweep: app is nil") + } + store, ok := app.Lookup[bouncer.BlacklistStore]() + if !ok || store == nil { + sqlDB, _, err := lagoon.OpenFromApp(ctx, app) + if err != nil { + return fmt.Errorf("user:jwt-blacklist-sweep: %w", err) + } + defer sqlDB.Close() + store = bouncer.NewPostgresBlacklist(sqlDB, classes.JWTBlacklistTable) + } + if err := store.Sweep(ctx, time.Now()); err != nil { + return fmt.Errorf("user:jwt-blacklist-sweep: %w", err) + } + out.Success("Deleted the expired JWT blacklist rows.") + return nil + }, + } +} diff --git a/logger.go b/logger.go deleted file mode 100644 index 19e6846..0000000 --- a/logger.go +++ /dev/null @@ -1,7 +0,0 @@ -package user - -import "git.golem15.com/golem15/summercms/modules/diary" - -// pluginLog is the plugin's logger: its records carry component=user, so -// config/log.yaml can raise or lower them (log.components.user). -var pluginLog = diary.For("user") diff --git a/plugin.go b/plugin.go index 9d0731d..e276122 100644 --- a/plugin.go +++ b/plugin.go @@ -38,6 +38,7 @@ var ( _ pact.HasMailTemplates = (*Plugin)(nil) _ pact.HasLang = (*Plugin)(nil) _ pact.HasCommands = (*Plugin)(nil) + _ pact.HasSchedule = (*Plugin)(nil) _ surf.BucketProvider = (*Plugin)(nil) _ bouncer.UserProvider = classes.GormUsers{} ) @@ -53,8 +54,7 @@ var langFS embed.FS // Plugin is the golem15.user plugin. type Plugin struct { - app *backpack.App - sweepStop chan struct{} + app *backpack.App } func (p *Plugin) ID() string { return "golem15.user" } @@ -79,19 +79,19 @@ func (p *Plugin) Boot(app *backpack.App) error { if _, err := classes.MailFallbackLocale(app); err != nil { return err } - // Serve publishes the database after Boot, so the GORM hooks, the - // blacklist store and its sweep are installed on lagoon.Publish, or at - // once when the database is already published. + if _, err := classes.BlacklistSweepInterval(app); err != nil { + return err + } + // Serve publishes the database after Boot, so the GORM hooks and the + // blacklist store are installed on lagoon.Publish, or at once when the + // database is already published. Expired blacklist rows are deleted by + // the scheduled user:jwt-blacklist-sweep command (Schedule). if err := lagoon.OnDatabase(app, func(sqlDB *sql.DB, gdb *gorm.DB) error { if err := classes.RegisterHooks(gdb); err != nil { return err } - store := bouncer.NewPostgresBlacklist(sqlDB, "jwt_blacklist") - if err := app.Publish[bouncer.BlacklistStore](store); err != nil { - return err - } - p.startBlacklistSweep(store) - return nil + store := bouncer.NewPostgresBlacklist(sqlDB, classes.JWTBlacklistTable) + return app.Publish[bouncer.BlacklistStore](store) }); err != nil { return err } @@ -221,37 +221,16 @@ func (p *Plugin) bootBackendLinks(app *backpack.App) error { return nil } -func (p *Plugin) startBlacklistSweep(bl bouncer.BlacklistStore) { - interval := 10 * time.Minute - if p.app != nil && p.app.Config != nil { - raw := p.app.Config.String("golem15.user.jwt.blacklist_sweep_interval") - if raw != "" { - parsed, err := time.ParseDuration(raw) - if err != nil || parsed <= 0 { - return - } - interval = parsed - } +// Schedule returns the scheduled commands: user:jwt-blacklist-sweep every +// golem15.user.jwt.blacklist_sweep_interval (10m by default), none when the +// interval is 0. p.app is nil before Register, which yields the default. A +// bad interval schedules nothing; Boot reports it. +func (p *Plugin) Schedule() []pact.ScheduledCommand { + interval, err := classes.BlacklistSweepInterval(p.app) + if err != nil || interval == 0 { + return nil } - if p.sweepStop != nil { - return - } - p.sweepStop = make(chan struct{}) - stop := p.sweepStop - go func() { - ticker := time.NewTicker(interval) - defer ticker.Stop() - for { - select { - case <-ticker.C: - if err := bl.Sweep(context.Background(), time.Now()); err != nil { - pluginLog.Error("golem15.user: jwt blacklist sweep", "err", err) - } - case <-stop: - return - } - } - }() + return []pact.ScheduledCommand{{Command: "user:jwt-blacklist-sweep", Cadence: pact.Every(interval)}} } func (p *Plugin) Buckets() map[string]surf.Bucket { @@ -312,8 +291,14 @@ func (p *Plugin) MailLayouts() map[string]string { } } +// Commands returns the console commands, among them +// user:jwt-blacklist-sweep, which Schedule runs. func (p *Plugin) Commands() []bonfire.Command { - return append([]bonfire.Command{console.RequirePasswordChange(p.withDB), console.BackfillHandles(p.app)}, console.APITokenCommands(p.app, p.withDB)...) + return append([]bonfire.Command{ + console.RequirePasswordChange(p.withDB), + console.BackfillHandles(p.app), + console.JWTBlacklistSweep(p.app), + }, console.APITokenCommands(p.app, p.withDB)...) } // withDB runs fn with the application's database: the one the serve command diff --git a/session_test.go b/session_test.go index 8e8f12c..5125e6d 100644 --- a/session_test.go +++ b/session_test.go @@ -506,47 +506,6 @@ func TestLoginRouteGroup(t *testing.T) { } } -func TestLoginBlacklistSweep(t *testing.T) { - gdb, sqlDB := sessionDB(t) - app := newSessionApp(t, gdb, sqlDB) - _, stop := bootPlugin(t, app, "50ms") - past := time.Now().Add(-time.Minute) - if err := gdb.Exec(`INSERT INTO jwt_blacklist (jti, expires_at, valid_until) VALUES (?, ?, ?)`, "sweep-me", past, past).Error; err != nil { - t.Fatal(err) - } - deadline := time.Now().Add(2 * time.Second) - for { - var n int64 - if err := gdb.Raw(`SELECT COUNT(*) FROM jwt_blacklist WHERE jti = ?`, "sweep-me").Scan(&n).Error; err != nil { - t.Fatal(err) - } - if n == 0 { - break - } - if time.Now().After(deadline) { - t.Fatal("expired jwt_blacklist row was not swept") - } - time.Sleep(20 * time.Millisecond) - } - stop() - time.Sleep(150 * time.Millisecond) - - quiet := newSessionApp(t, gdb, sqlDB) - bootPlugin(t, quiet, "0s") - later := time.Now().Add(-time.Minute) - if err := gdb.Exec(`INSERT INTO jwt_blacklist (jti, expires_at, valid_until) VALUES (?, ?, ?)`, "sweep-keep", later, later).Error; err != nil { - t.Fatal(err) - } - time.Sleep(200 * time.Millisecond) - var n int64 - if err := gdb.Raw(`SELECT COUNT(*) FROM jwt_blacklist WHERE jti = ?`, "sweep-keep").Scan(&n).Error; err != nil { - t.Fatal(err) - } - if n != 1 { - t.Fatal("non-positive sweep interval removed a row") - } -} - type captureMail struct { got []string } @@ -707,7 +666,7 @@ func newSessionApp(t *testing.T, gdb *gorm.DB, sqlDB *sql.DB) *backpack.App { return app } -func bootPlugin(t *testing.T, app *backpack.App, sweep string) (*Plugin, func()) { +func bootPlugin(t *testing.T, app *backpack.App, sweep string) *Plugin { t.Helper() if sweep != "" { if err := app.Config.Set("golem15.user.jwt.blacklist_sweep_interval", sweep); err != nil { @@ -718,14 +677,7 @@ func bootPlugin(t *testing.T, app *backpack.App, sweep string) (*Plugin, func()) if err := p.Boot(app); err != nil { t.Fatal(err) } - stop := func() {} - if p.sweepStop != nil { - ch := p.sweepStop - var once sync.Once - stop = func() { once.Do(func() { close(ch) }) } - t.Cleanup(stop) - } - return p, stop + return p } func sessionConfig(t *testing.T) *compass.Config {