feat(16.2-01): translatable plugin field types, untouched skip, writes, list values and uploads

- translatable types lift, store and hydrate through the ML path
  (isMLField); Decode and Load run per locale and notices carry the locale
- on update a value equal to the served value (Load of the stored host
  value or exact translation) is neither decoded nor written
- FieldWrite, FieldWritesFromContext and WithFieldWrites expose what a save
  writes to the Form hooks
- list columns may use a plugin alias and show the type's ListValue
- POST .../{id}/fields/{field}/upload hands the file to the type's Upload
  (guard, X-Requested-With, permissions, field context, record scope, body
  cap); OpenAPI, route inventories, README and docs updated
This commit is contained in:
2026-10-10 11:19:24 +02:00
parent 83fa2f7b2c
commit 8fa92db96d
21 changed files with 1281 additions and 67 deletions

View File

@@ -22,7 +22,7 @@ Schema-driven admin backend that compiles WinterCMS-style YAML list, form, filte
- Row state: a controller implementing `pact.ListRowStates` is called once per list page with the page's records and the list's database handle. The list response carries `meta.row_states`, keyed by row id, with values from the fixed set `deleted`, `negative`, `disabled` in that order; a value outside the set is dropped and logged, rows without a state are left out, and a controller without the hook sends no `row_states` key. The badge texts are the list messages `rowStateDeleted`, `rowStateNegative` and `rowStateDisabled`, defaulting to `backend::lang.messages.list.row_state_*`. A soft-deleted record that the controller's `pact.ListExtendQuery` and `pact.FormExtendQuery` include can be shown, updated (it stays soft-deleted), targeted by bulk and record actions and removed for good by the controller's `pact.FormAfterDelete`.
- Record actions: `recordActions` in `config_form.yaml` lists names the controller registers through `pact.HasAdminRecordActions`, a third action namespace with the same reserved names. It needs the form's `preview` block: record actions are offered on the preview screen, and a form that declares them without one fails boot. The show response's `meta.actions` (`cabana.RecordAction` entries with localized `label` and `confirm`) carries only the declared actions the requesting administrator may run and whose `Applies` reports true for the record; the key is absent when none is offered, and create and update responses never carry it. The action route loads the record through `pact.FormExtendQuery` with a row lock in one transaction (one 404 for a missing and an out-of-scope id), checks `Applies` again (409 when it reports false) and then runs the action. An unknown or duplicate name, or an action without a label, fails boot. Each run is logged with the controller, action, administrator and record id.
- Preview screen: a `preview` mapping in `config_form.yaml` (`preview: {}`, or with `headerPartial: <name>` for a status hint) gives the form a read-only record screen in the admin SPA. The form schema reports it as `preview` (`cabana.FormPreview`), fields with `context: preview` are shown only there and are never written by a save, `messages.preview` and `messages.edit` name the screen's subtitle and edit button, and `recordUrl` and the form redirects may point at it as `.../preview/:id`. An empty `preview:` key or an unknown key inside it fails boot.
- Plugin field types: a plugin implementing `pact.HasFormFieldTypes` registers admin form field types under short aliases that any controller's `fields.yaml` uses as `type:`. Boot checks the aliases (no built-in type, no alias claimed by two plugins), the element prefix, the scripts and styles and each field's option keys and `CheckOptions`. A save runs the type's `Decode` inside the save transaction after the record is loaded: the decoded value is stored, a `pact.FormFieldError` is a 422 on the field, any other error an opaque 500, and the notices a type returns are localized into the response's `meta.notices` (`cabana.FieldNotice`), which is absent when no type reported anything. Show and save responses serve the value through the type's `Load`. The form schema carries `element`, `translatable` and `typeOptions` for such a field. Settings and relation forms refuse plugin field types. See [Plugin field types](../../docs/backend/field-types.md).
- Plugin field types: a plugin implementing `pact.HasFormFieldTypes` registers admin form field types under short aliases that any controller's `fields.yaml` uses as `type:`. Boot checks the aliases (no built-in type, no alias claimed by two plugins), the element prefix, the scripts and styles and each field's option keys and `CheckOptions`. A save runs the type's `Decode` inside the save transaction after the record is loaded: the decoded value is stored, a `pact.FormFieldError` is a 422 on the field, any other error an opaque 500, and the notices a type returns are localized into the response's `meta.notices` (`cabana.FieldNotice`), which is absent when no type reported anything. Show and save responses serve the value through the type's `Load`. A translatable type's value is a locale map stored through the same lift, `cabana.TranslationWriter` and hydrate path as `mlmarkdown`, decoded and loaded per locale. On update, a value equal to the one the form was served for that field and locale is neither decoded nor written. Form hooks read what a save wrote through `cabana.FieldWritesFromContext`. A list column may name a plugin alias as its `type` and shows the type's `ListValue`. The cabana-owned upload route hands an element's file to the type's `Upload`. The form schema carries `element`, `translatable` and `typeOptions` for such a field. Settings and relation forms refuse plugin field types (supported in controller forms only). See [Plugin field types](../../docs/backend/field-types.md).
- Form-only fields: a controller implementing `pact.FormVirtualFields` lists fields of its `fields.yaml` that are not columns of the form. They are exempt from the column binding, never filled into the model and never part of a record response; the values an administrator submits reach the Form hooks through `cabana.VirtualFieldsFromContext`, only for fields whose `context` allows the operation, and a nested value is a 422 on the field. `type: password` is a masked field that must be listed this way, so a password is sent in a save body and never comes back. A controller implementing `pact.FormRules` supplies the validation rules per operation (`create` or `update`), which replace the model's `Rules()` for admin saves; a rule on a virtual field is checked against the submitted value, never against a model column of the same name. Neither is available on settings forms or relation forms.
- Permission editor: a `type: permissioneditor` field with `mode: radio` (allow `1`, inherit, deny `-1`) or `mode: checkbox` (allow `1`) edits a record's permission set as a JSON object of code to integer. The controller implements `cabana.PermissionEditorProvider`: it returns the offered `cabana.PermissionOption` list per request (served on the field as `permissionOptions`, with `locked` for permissions the administrator may not change) and reads and stores the record's values, so the storage shape is the plugin's. A save answers 422 on the field for a value that is not an object of integers, a code that is not offered or a value outside the mode's set, and 403 `forbidden` when a locked code's value changes; stored codes that are not offered are kept. The widget fill contract is unchanged: a widget still writes scalar fields only.
- Markdown and multilingual fields: `type: markdown` edits source on a host text column; `cabana.RenderMarkdown` turns that source into HTML with the pinned goldmark engine and no unsafe HTML, and leftover script or iframe tags, event handlers, or javascript, vbscript or data URLs are rejected; the admin form preview renders through it via POST `/markdown/preview`. `type: mltext`, `type: mltextarea` and `type: mlmarkdown` take a JSON object of locale code to string; `mltextarea` is the multi-line variant and takes `size` like `textarea`. The form schema's `cabana.FormMeta.EnabledLocales` lists every enabled code when a `cabana.TranslationWriter` is published, default locale first and then the other enabled codes in the writer's order, so every ML control opens on the locale stored in the host columns; create seeds an empty string per code, and Show/save replace the host scalar with that map via `hydrateMLRecord` and `TranslationWriter.TranslatedExact` (missing non-default codes stay empty; D-11 fallback is not applied). A GET host string is merged onto the seed so sibling locales are not dropped. The default locale fills the host column; other locales reach a plugin-published `cabana.TranslationWriter` after the host row has a primary key, still inside the controller's permissioned save transaction. Relation-child create, update and show on `cabana.RelationService` use the same lift, apply and `hydrateMLRecord` path as controller save, still with no standalone translate-write route. An unknown field type, including an unknown `ml*` type, fails boot.
@@ -52,6 +52,7 @@ All paths are relative to `<prefix>/api/v1`. A controller ID `vendor.plugin.cont
| POST and DELETE `/account/avatar` | Upload the signed-in administrator's avatar (one multipart `file_data` part: a jpg, jpeg, png, gif or webp image of at most 2 MiB; anything else is a 422 on `avatar`, a body over the cap a 413) replacing the previous one, or remove it (also 200 when there is none). Both answer the `cabana.AdminProfile`. Needs any backend session. |
| GET `/navigation`, GET `/settings` | Navigation and settings entries the administrator may open; a settings entry's `controller` names the admin controller a link entry opens and is empty for a singleton form. |
| GET `/field-types` | The plugin field types (`pact.HasFormFieldTypes`) sorted by name, each with its `element`, `translatable` flag and the versioned URLs of its `scripts` and `styles` (`cabana.FieldTypesResponse`); an empty list when no plugin registers one. The SPA loads them after sign-in. Needs any backend session. |
| POST `/{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload` | Store one multipart part named `file` through the `Upload` of the field's plugin field type and answer 201 `{url}` (`cabana.FieldUploadResponse`). `{id}` 0 is the create form and passes no record; any other id loads the record through the form scope (404 outside it). 404 for a field without a plugin type with `Upload` or whose context does not allow the operation, 413 over `http.body_limits.upload_bytes`, 422 on the field for a malformed body or a `pact.FormFieldError`. |
| GET `/settings/{code}/schema`, GET and PUT `/settings/{code}` | Settings form schema, values and update. |
| POST `/markdown/preview` | Render `{markdown}` through `cabana.RenderMarkdown` for the form preview and answer `{html}`; a refused output is a 422 `validation_failed` on `markdown`. Needs any backend session. |
| GET `/{vendor}/{plugin}/{controller}/schema/list`, `.../schema/form`, `.../schema/relation/{name}` | Localized list, form and relation schemas. |
@@ -234,6 +235,10 @@ func (p *Plugin) AdminFS() fs.FS { return adminFS }
| `cabana.Allows` | Checks a principal against required permission codes. |
| `cabana.TxFromContext` | The transaction a write route is running in, from the context of a lifecycle hook or scope. |
| `cabana.VirtualFieldsFromContext` | The submitted values of the form's virtual fields (`pact.FormVirtualFields`), from the context of a Form hook during a create or update; a copy, keyed by field name. |
| `cabana.FieldWrite` | One value a save wrote for a plugin field type: the content `Locale` (translatable types), `Default` (written to the host column) and the decoded `Value`. |
| `cabana.FieldWritesFromContext` | The values a create or update save writes for the form's plugin field types, per field and locale, from the context of a Form hook; untouched values are not listed. A copy; false outside a save. |
| `cabana.WithFieldWrites` | Puts field writes on a context as a save does, for tests of Form hooks that read `cabana.FieldWritesFromContext`. |
| `cabana.FieldUploadResponse` / `cabana.AdminFieldUpload` | Answer (`url`) and swag annotation of the field upload route. |
| `cabana.PermissionOption` | One permission a `type: permissioneditor` field offers: code, label, optional tab and comment, and `Locked`. |
| `cabana.PermissionEditorProvider` | Controller capability behind a `type: permissioneditor` field: `AdminPermissionOptions`, `AdminPermissionValues` and `AdminSetPermissionValues`. |
| `cabana.FormMeta` | Form (and optional list) schema meta: `Locale` is the UI locale of the response; `EnabledLocales` lists content locales for ML fields when a `cabana.TranslationWriter` is published, default locale first and then the other enabled codes in the writer's order (`json:"enabledLocales,omitempty"` so list schemas omit the key). |
@@ -302,7 +307,7 @@ printf '%s\n' "$ADMIN_PASSWORD" | ./bin/acme admin:reset-password admin@example.
- SummerCMS modules: [backpack](../backpack/README.md), [boardwalk](../boardwalk/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [lagoon](../lagoon/README.md), [pact](../pact/README.md), [party](../party/README.md), [phrasebook](../phrasebook/README.md), [towel](../towel/README.md).
- Third-party: `gorm.io/gorm` (with `gorm.io/gorm/clause`), `github.com/goccy/go-yaml` (with its `ast` package), `golang.org/x/net/html` (with its `atom` package; parses rendered partials for the allowlist walk).
- Standard library: `bytes`, `context`, `crypto/sha256`, `database/sql`, `encoding/hex`, `encoding/json`, `errors`, `fmt`, `html/template`, `io`, `io/fs`, `log/slog`, `math`, `net`, `net/http`, `path`, `reflect`, `regexp`, `sort`, `strconv`, `strings`, `time`.
- Standard library: `bytes`, `context`, `crypto/sha256`, `database/sql`, `encoding/hex`, `encoding/json`, `errors`, `fmt`, `html/template`, `io`, `io/fs`, `log/slog`, `math`, `net`, `net/http`, `path`, `reflect`, `regexp`, `sort`, `strconv`, `strings`, `time`, `unicode/utf8`.
- Tests additionally use `github.com/testcontainers/testcontainers-go` and its `modules/postgres` package.
## Testing

View File

@@ -657,7 +657,7 @@ func AdminShow() {}
// AdminUpdate documents the record update route.
//
// @Summary Update an admin record
// @Description A plugin field type's value is decoded by its type inside the save. When a type reports what it changed, meta.notices lists the localized messages per field (and content locale), and the key is absent otherwise.
// @Description A plugin field type's value is decoded by its type inside the save; a value equal to the one the form was served for that field and locale is neither decoded nor written. When a type reports what it changed, meta.notices lists the localized messages per field (and content locale), and the key is absent otherwise.
// @Tags admin
// @Accept json
// @Produce json
@@ -1171,6 +1171,29 @@ func AdminFileList() {}
// @Router /{vendor}/{plugin}/{controller}/{id}/files/{field} [post]
func AdminFileUpload() {}
// AdminFieldUpload documents the upload route of a plugin field type.
//
// @Summary Upload a file for a plugin field type
// @Description Stores one multipart part named file through the Upload of the field's plugin field type (pact.FormFieldType) and answers its URL. id 0 is the create form and passes no record; any other id loads the record through the controller's FormExtendQuery scope. A field that is not in the form, has a built-in type, has a type without Upload or whose context does not allow the operation answers 404, as does a record outside the scope. A body over http.body_limits.upload_bytes answers 413; a malformed body, a second part or a value the type refuses (pact.FormFieldError) answers 422 on the field.
// @Tags admin
// @Accept multipart/form-data
// @Produce json
// @Security BackendBearer
// @Param vendor path string true "Vendor"
// @Param plugin path string true "Plugin"
// @Param controller path string true "Controller"
// @Param id path integer true "Record id (0 for the create form)"
// @Param field path string true "Field name"
// @Param file formData file true "The file"
// @Success 201 {object} Envelope[FieldUploadResponse]
// @Failure 401 {object} ErrorEnvelope
// @Failure 403 {object} ErrorEnvelope
// @Failure 404 {object} ErrorEnvelope
// @Failure 413 {object} ErrorEnvelope
// @Failure 422 {object} ErrorEnvelope
// @Router /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload [post]
func AdminFieldUpload() {}
// AdminFileUpdate documents the caption route of a fileupload field.
//
// @Summary Save a file's title and description

View File

@@ -444,7 +444,7 @@ func (s CRUDService) ShowRecord(ctx context.Context, cc *CompiledController, id
if err := hydrateMLRecord(ctx, tx, cc, s.writer, target, result.Data, "update"); err != nil {
return err
}
if err := hydratePluginFields(ctx, tx, cc, target, result.Data, "update"); err != nil {
if err := hydratePluginFields(ctx, tx, cc, s.writer, target, result.Data, "update"); err != nil {
return err
}
result.Meta.Actions, err = s.offeredRecordActions(withTx(ctx, tx), cc, target)
@@ -627,10 +627,13 @@ func (s CRUDService) save(ctx context.Context, cc *CompiledController, id any, i
}
}
projected := projectOperation(cc, in.Body, op)
notices, err = decodePluginFields(ctx, tx, s.tr, cc, target, pluginValues, projected)
decoded, err := decodePluginFields(ctx, pluginDecodeInput{Tx: tx, Tr: s.tr, Writer: s.writer, CC: cc, Target: target, Values: pluginValues, Projected: projected, Translations: translations, Update: update})
if err != nil {
return err
}
notices = decoded.notices
// The Form hooks read what this save writes for plugin field types.
ctx = withFieldWrites(ctx, decoded.writes)
if err := lagoon.Fill(target, fillAllowed(cc, target, op), projected, false); err != nil {
// A value that does not fit its column is the admin's input,
// not a missing capability: answer it on the field.
@@ -722,7 +725,7 @@ func (s CRUDService) save(ctx context.Context, cc *CompiledController, id any, i
if err := hydrateMLRecord(ctx, tx, cc, s.writer, target, result.Data, op); err != nil {
return err
}
return hydratePluginFields(ctx, tx, cc, target, result.Data, op)
return hydratePluginFields(ctx, tx, cc, s.writer, target, result.Data, op)
})
if err != nil {
return RecordResult{}, err

View File

@@ -52,6 +52,13 @@ func mlFieldType(typ string) bool {
return false
}
// isMLField reports whether a form field's value is a locale map stored
// through the translation writer: a built-in ML type (mlFieldType) or a
// translatable plugin field type (pact.FormFieldType.Translatable).
func isMLField(field FormField) bool {
return mlFieldType(field.Type) || (field.fieldType != nil && field.fieldType.def.Translatable)
}
// liftMLValues takes declared mltext/mltextarea/mlmarkdown maps out of a save body
// before ProjectWritableFields drops nested values. Only fields present in
// the body whose context allows op are lifted. A value must be a JSON object
@@ -64,7 +71,7 @@ func liftMLValues(ctx context.Context, cc *CompiledController, body map[string]a
}
var names []string
for _, field := range cc.Form.Fields {
if !mlFieldType(field.Type) || !contextAllows(cc, field.Name, op) {
if !isMLField(field) || !contextAllows(cc, field.Name, op) {
continue
}
if _, present := body[field.Name]; present {
@@ -186,7 +193,7 @@ func hydrateMLRecord(ctx context.Context, tx *gorm.DB, cc *CompiledController, w
}
var names []string
for _, field := range cc.Form.Fields {
if !mlFieldType(field.Type) || !contextAllows(cc, field.Name, op) {
if !isMLField(field) || !contextAllows(cc, field.Name, op) {
continue
}
names = append(names, field.Name)

View File

@@ -1,7 +1,9 @@
package cabana
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
@@ -396,42 +398,256 @@ func liftPluginValues(cc *CompiledController, body map[string]any, op string) ma
return out
}
// pluginDecodeInput is what decodePluginFields works on: the save's
// transaction, translator and translation writer, the loaded (update) or new
// (create) record, the lifted values of plain plugin fields, the projected
// body Fill receives and the lifted ML translations.
type pluginDecodeInput struct {
Tx *gorm.DB
Tr *phrasebook.Translator
Writer TranslationWriter
CC *CompiledController
Target any
Values map[string]any
Projected map[string]any
Translations []mlValue
Update bool
}
// pluginDecodeResult is the notices the types reported and the values the
// save writes, per field.
type pluginDecodeResult struct {
notices []pluginNotice
writes map[string][]FieldWrite
}
// decodePluginFields runs each submitted plugin field value through its
// type's Decode inside the save transaction, after the record is loaded and
// before Fill, and stores the decoded value in projected. It returns the
// notices the types reported. A *pact.FormFieldError is validation_failed on
// the field; any other error is the opaque lifecycle error.
func decodePluginFields(ctx context.Context, tx *gorm.DB, tr *phrasebook.Translator, cc *CompiledController, target any, values, projected map[string]any) ([]pluginNotice, error) {
if cc == nil || cc.Form == nil || len(values) == 0 {
return nil, nil
// before Fill. A plain type's value comes from Values and its decoded value
// goes to Projected. A translatable type is decoded per locale: the default
// locale's value is the projected host value, every other locale's value is
// the matching entry of Translations, which applyMLTranslations writes. On
// update, a value equal to the value Show serves for that field and locale
// (servedFieldValue) is neither decoded nor written. A *pact.FormFieldError
// is validation_failed on the field and discards every notice; any other
// error is the opaque lifecycle error.
func decodePluginFields(ctx context.Context, in pluginDecodeInput) (pluginDecodeResult, error) {
out := pluginDecodeResult{writes: map[string][]FieldWrite{}}
cc := in.CC
if cc == nil || cc.Form == nil {
return out, nil
}
defaultLocale := ""
defaultKnown := false
resolveDefault := func() (string, error) {
if defaultKnown || in.Writer == nil {
return defaultLocale, nil
}
code, err := in.Writer.DefaultLocale(ctx, in.Tx)
if err != nil {
return "", fmt.Errorf("cabana: default locale: %w", err)
}
defaultLocale, defaultKnown = code, true
return defaultLocale, nil
}
var notices []pluginNotice
for _, field := range cc.Form.Fields {
ft := field.fieldType
if ft == nil {
continue
}
value, ok := values[field.Name]
if !ok {
if !ft.def.Translatable {
value, ok := in.Values[field.Name]
if !ok {
continue
}
stored, keep, err := decodeOne(ctx, in, field, "", true, value, &out)
if err != nil {
return pluginDecodeResult{}, err
}
if keep {
in.Projected[field.Name] = stored
}
continue
}
out, err := ft.def.Decode(ctx, pact.FormFieldValue{
DB: tx,
Record: target,
Field: field.Name,
Options: copyJSONMap(field.TypeOptions),
Default: true,
Value: value,
})
def, err := resolveDefault()
if err != nil {
return nil, fieldTypeFailure(ctx, tr, cc, field, "decode", err)
return pluginDecodeResult{}, err
}
projected[field.Name] = out.Value
for _, notice := range out.Notices {
notices = append(notices, pluginNotice{field: field.Name, notice: notice})
if value, ok := in.Projected[field.Name]; ok {
stored, keep, err := decodeOne(ctx, in, field, def, true, value, &out)
if err != nil {
return pluginDecodeResult{}, err
}
if keep {
in.Projected[field.Name] = stored
} else {
delete(in.Projected, field.Name)
}
}
for _, item := range in.Translations {
if item.field != field.Name {
continue
}
locales := make([]string, 0, len(item.values))
for locale := range item.values {
if locale != def {
locales = append(locales, locale)
}
}
sort.Strings(locales)
for _, locale := range locales {
stored, keep, err := decodeOne(ctx, in, field, locale, false, item.values[locale], &out)
if err != nil {
return pluginDecodeResult{}, err
}
if keep {
item.values[locale] = stored.(string)
} else {
delete(item.values, locale)
}
}
}
}
return notices, nil
if len(out.writes) == 0 {
out.writes = map[string][]FieldWrite{}
}
return out, nil
}
// decodeOne decodes one value of a plugin field in one locale. keep is false
// when the value is skipped as untouched (update only). It records the
// notices and the write on out.
func decodeOne(ctx context.Context, in pluginDecodeInput, field FormField, locale string, isDefault bool, value any, out *pluginDecodeResult) (any, bool, error) {
ft := field.fieldType
if in.Update {
served, err := servedFieldValue(ctx, in.Tx, in.CC, in.Writer, in.Target, field, locale, isDefault)
if err != nil {
return nil, false, err
}
if sameJSON(value, served) {
return nil, false, nil
}
}
decoded, err := ft.def.Decode(ctx, pact.FormFieldValue{
DB: in.Tx,
Record: in.Target,
Field: field.Name,
Options: copyJSONMap(field.TypeOptions),
Locale: locale,
Default: isDefault,
Value: value,
Translated: translatedReader(ctx, in.Tx, in.Writer, in.Target, ft, locale),
})
if err != nil {
return nil, false, fieldTypeFailure(ctx, in.Tr, in.CC, field, "decode", err)
}
if ft.def.Translatable {
if _, ok := decoded.Value.(string); !ok {
return nil, false, opaqueFieldTypeFailure(in.CC, field, "decode", fmt.Errorf("translatable type returned %T, want a string", decoded.Value))
}
}
for _, notice := range decoded.Notices {
out.notices = append(out.notices, pluginNotice{field: field.Name, locale: locale, notice: notice})
}
out.writes[field.Name] = append(out.writes[field.Name], FieldWrite{Locale: locale, Default: isDefault, Value: decoded.Value})
return decoded.Value, true, nil
}
// translatedReader is FormFieldValue.Translated: an exact read of another
// attribute of the record in locale through the translation writer. It is
// nil for a type that is not translatable and without a writer.
func translatedReader(ctx context.Context, tx *gorm.DB, writer TranslationWriter, target any, ft *compiledFieldType, locale string) func(string) (string, bool, error) {
if writer == nil || ft == nil || !ft.def.Translatable {
return nil
}
return func(attribute string) (string, bool, error) {
return writer.TranslatedExact(ctx, tx, target, attribute, locale)
}
}
// servedFieldValue is what Show serves for a plugin field in one locale of
// the loaded record: the host column read through the field's column binding
// (pointers dereferenced; a translatable type's host value as text), or the
// exact stored translation of a non-default locale ("" when missing), then
// the type's Load.
func servedFieldValue(ctx context.Context, tx *gorm.DB, cc *CompiledController, writer TranslationWriter, target any, field FormField, locale string, isDefault bool) (any, error) {
ft := field.fieldType
var stored any
if isDefault {
stored = hostFieldValue(cc, target, field.Name)
if ft.def.Translatable {
stored = hostScalarString(stored)
}
} else if writer != nil {
text, ok, err := writer.TranslatedExact(ctx, tx, target, field.Name, locale)
if err != nil {
return nil, err
}
if ok {
stored = text
} else {
stored = ""
}
} else {
stored = ""
}
if ft.def.Load == nil {
return stored, nil
}
value, err := ft.def.Load(ctx, pact.FormFieldValue{
DB: tx,
Record: target,
Field: field.Name,
Options: copyJSONMap(field.TypeOptions),
Locale: locale,
Default: isDefault,
Value: stored,
Translated: translatedReader(ctx, tx, writer, target, ft, locale),
})
if err != nil {
return nil, opaqueFieldTypeFailure(cc, field, "load", err)
}
return value, nil
}
// hostFieldValue reads a field's column on the record through the binding
// BindWritableFields made (the fill key), with pointers dereferenced.
func hostFieldValue(cc *CompiledController, target any, name string) any {
key := name
if cc != nil {
for _, field := range cc.Writable {
if field.Name == name {
key = field.FillKey
break
}
}
}
v := reflect.ValueOf(target)
for v.Kind() == reflect.Pointer {
if v.IsNil() {
return nil
}
v = v.Elem()
}
column := fieldByColumn(v, key)
if !column.IsValid() || !column.CanInterface() {
return nil
}
return derefValue(column.Interface())
}
// sameJSON reports whether a and b have the same canonical encoding/json
// form (object keys sorted).
func sameJSON(a, b any) bool {
left, err := json.Marshal(a)
if err != nil {
return false
}
right, err := json.Marshal(b)
if err != nil {
return false
}
return bytes.Equal(left, right)
}
// fieldTypeFailure classifies a plugin field type's error: a
@@ -465,11 +681,15 @@ func opaqueFieldTypeFailure(cc *CompiledController, field FormField, stage strin
}
// hydratePluginFields serves each plugin field of a Show or save response
// through its type's Load. A type without Load serves the stored value.
func hydratePluginFields(ctx context.Context, tx *gorm.DB, cc *CompiledController, target any, data map[string]any, op string) error {
// through its type's Load: a translatable type per locale of the hydrated
// locale map, any other type once. A type without Load serves the stored
// value.
func hydratePluginFields(ctx context.Context, tx *gorm.DB, cc *CompiledController, writer TranslationWriter, target any, data map[string]any, op string) error {
if cc == nil || cc.Form == nil || data == nil {
return nil
}
defaultLocale := ""
defaultKnown := false
for _, field := range cc.Form.Fields {
ft := field.fieldType
if ft == nil || ft.def.Load == nil || !contextAllows(cc, field.Name, op) {
@@ -479,20 +699,49 @@ func hydratePluginFields(ctx context.Context, tx *gorm.DB, cc *CompiledControlle
if !ok {
continue
}
value, err := ft.def.Load(ctx, pact.FormFieldValue{
DB: tx,
Record: target,
Field: field.Name,
Options: copyJSONMap(field.TypeOptions),
Default: true,
Value: derefValue(stored),
})
if err != nil {
// A Load error is never a 422, even a pact.FormFieldError:
// nothing was submitted.
return opaqueFieldTypeFailure(cc, field, "load", err)
load := func(value any, locale string, isDefault bool) (any, error) {
out, err := ft.def.Load(ctx, pact.FormFieldValue{
DB: tx,
Record: target,
Field: field.Name,
Options: copyJSONMap(field.TypeOptions),
Locale: locale,
Default: isDefault,
Value: value,
Translated: translatedReader(ctx, tx, writer, target, ft, locale),
})
if err != nil {
// A Load error is never a 422, even a pact.FormFieldError:
// nothing was submitted.
return nil, opaqueFieldTypeFailure(cc, field, "load", err)
}
return out, nil
}
data[field.Name] = value
texts, isMap := stored.(map[string]string)
if !ft.def.Translatable || !isMap {
value, err := load(derefValue(stored), "", true)
if err != nil {
return err
}
data[field.Name] = value
continue
}
if !defaultKnown && writer != nil {
code, err := writer.DefaultLocale(ctx, tx)
if err != nil {
return fmt.Errorf("cabana: default locale: %w", err)
}
defaultLocale, defaultKnown = code, true
}
served := make(map[string]any, len(texts))
for locale, text := range texts {
value, err := load(text, locale, locale == defaultLocale)
if err != nil {
return err
}
served[locale] = value
}
data[field.Name] = served
}
return nil
}
@@ -610,3 +859,9 @@ func (s *service) fieldTypesList(w http.ResponseWriter, r *http.Request) {
}
WriteData(w, http.StatusOK, s.fieldTypeAssets(), nil)
}
// FieldUploadResponse is the answer of the field upload route: the URL of
// the stored file, from the type's Upload.
type FieldUploadResponse struct {
URL string `json:"url"`
}

View File

@@ -1,10 +1,14 @@
package cabana
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"mime/multipart"
"net/http"
"net/http/httptest"
"strings"
@@ -35,6 +39,9 @@ func (ftController) ID() string { return "acme.demo.notes" }
func (ftController) ModelName() string { return "Note" }
func (ftController) ConfigDir() string { return "controllers/notes" }
func (ftController) NewRecord() any { return &ftNote{} }
func (ftController) RequiredPermissions() []string {
return []string{"acme.demo.access"}
}
// ftPlugin registers field types and serves an AdminFS.
type ftPlugin struct {
@@ -67,13 +74,13 @@ recordsPerPage: 20
// ftFS is the plugin tree: the notes script, the controller's list and form,
// and fields.yaml with the given notes field body.
func ftFS(notesField string) fstest.MapFS {
func ftFS(notesField string, extra ...string) fstest.MapFS {
return fstest.MapFS{
"assets/js/notes.js": &fstest.MapFile{Data: []byte("customElements.define('acme-demo-notes', class extends HTMLElement {})\n")},
"controllers/notes/config_form.yaml": &fstest.MapFile{Data: []byte(ftFormConfig)},
"controllers/notes/config_list.yaml": &fstest.MapFile{Data: []byte(ftListConfig)},
"models/note/columns.yaml": &fstest.MapFile{Data: []byte("columns:\n title:\n label: Title\n")},
"models/note/fields.yaml": &fstest.MapFile{Data: []byte("fields:\n title:\n type: text\n label: Title\n notes:\n" + notesField)},
"models/note/fields.yaml": &fstest.MapFile{Data: []byte("fields:\n title:\n type: text\n label: Title\n notes:\n" + notesField + strings.Join(extra, ""))},
}
}
@@ -102,6 +109,20 @@ func ftNotesType() pact.FormFieldType {
text, _ := in.Value.(string)
return "served:" + text, nil
},
Upload: func(_ context.Context, in pact.FormFieldUpload) (pact.FormFieldUploadResult, error) {
body, err := io.ReadAll(in.Body)
if err != nil {
return pact.FormFieldUploadResult{}, err
}
if in.Filename == "bad.txt" {
return pact.FormFieldUploadResult{}, &pact.FormFieldError{Messages: []string{"acme.demo::lang.bad_file"}}
}
record := "nil"
if in.Record != nil {
record = "set"
}
return pact.FormFieldUploadResult{URL: fmt.Sprintf("/media/%s?bytes=%d&record=%s&rows=%v", in.Filename, len(body), record, in.Options["rows"])}, nil
},
}
}
@@ -119,6 +140,11 @@ func ftCompile(plugins ...party.Plugin) (*Registry, error) {
return compileRegistryWithTypes(items, types)
}
// ftDecode decodes values on a create with no translation writer.
func ftDecode(cc *CompiledController, target any, values, projected map[string]any) (pluginDecodeResult, error) {
return decodePluginFields(context.Background(), pluginDecodeInput{CC: cc, Target: target, Values: values, Projected: projected})
}
func ftDemo(notesField string, types ...pact.FormFieldType) ftPlugin {
if len(types) == 0 {
types = []pact.FormFieldType{ftNotesType()}
@@ -265,19 +291,19 @@ func TestFieldTypeTracerSmoke(t *testing.T) {
t.Run("decode stores the type's value", func(t *testing.T) {
projected := map[string]any{}
notices, err := decodePluginFields(context.Background(), nil, nil, cc, &ftNote{}, map[string]any{"notes": "plain"}, projected)
if err != nil || projected["notes"] != "plain" || len(notices) != 0 {
t.Fatalf("projected=%v notices=%v err=%v", projected, notices, err)
res, err := ftDecode(cc, &ftNote{}, map[string]any{"notes": "plain"}, projected)
if err != nil || projected["notes"] != "plain" || len(res.notices) != 0 {
t.Fatalf("projected=%v notices=%v err=%v", projected, res.notices, err)
}
})
t.Run("decode notice becomes a FieldNotice", func(t *testing.T) {
projected := map[string]any{}
notices, err := decodePluginFields(context.Background(), nil, nil, cc, &ftNote{}, map[string]any{"notes": " x "}, projected)
res, err := ftDecode(cc, &ftNote{}, map[string]any{"notes": " x "}, projected)
if err != nil || projected["notes"] != "x" {
t.Fatalf("projected=%v err=%v", projected, err)
}
got := localizeNotices(context.Background(), nil, notices)
got := localizeNotices(context.Background(), nil, res.notices)
if len(got) != 1 || got[0] != (FieldNotice{Field: "notes", Message: "acme.demo::lang.trimmed"}) {
t.Fatalf("notices=%+v", got)
}
@@ -295,7 +321,7 @@ func TestFieldTypeTracerSmoke(t *testing.T) {
})
t.Run("FormFieldError is validation_failed on the field", func(t *testing.T) {
_, err := decodePluginFields(context.Background(), nil, nil, cc, &ftNote{}, map[string]any{"notes": "bad"}, map[string]any{})
_, err := ftDecode(cc, &ftNote{}, map[string]any{"notes": "bad"}, map[string]any{})
var invalid *ValidationError
if !errors.As(err, &invalid) {
t.Fatalf("err=%v, want a ValidationError", err)
@@ -315,7 +341,7 @@ func TestFieldTypeTracerSmoke(t *testing.T) {
t.Fatal(err)
}
occ, _ := other.Get("acme.demo.notes")
_, err = decodePluginFields(context.Background(), nil, nil, occ, &ftNote{}, map[string]any{"notes": "x"}, map[string]any{})
_, err = ftDecode(occ, &ftNote{}, map[string]any{"notes": "x"}, map[string]any{})
var life *lifecycleError
if !errors.As(err, &life) || strings.Contains(err.Error(), "secret") {
t.Fatalf("err=%v, want the opaque lifecycle error", err)
@@ -329,7 +355,7 @@ func TestFieldTypeTracerSmoke(t *testing.T) {
t.Run("show serves the value through Load", func(t *testing.T) {
data := projectRecord(cc, &ftNote{ID: 7, Notes: "stored"})
if err := hydratePluginFields(context.Background(), nil, cc, &ftNote{}, data, "update"); err != nil {
if err := hydratePluginFields(context.Background(), nil, cc, nil, &ftNote{}, data, "update"); err != nil {
t.Fatal(err)
}
if data["notes"] != "served:stored" || data["title"] != "" {
@@ -348,3 +374,341 @@ func TestFieldTypeTracerSmoke(t *testing.T) {
}
})
}
// ftNotesMLType is a translatable type: Decode refuses "bad", trims and
// reports the trim per locale; ListValue prefixes "list:". It has no Load
// and no Upload.
func ftNotesMLType() pact.FormFieldType {
return pact.FormFieldType{
Name: "notesml",
Element: "acme-demo-notesml",
Translatable: true,
Scripts: []string{"assets/js/notes.js"},
Decode: func(_ context.Context, in pact.FormFieldValue) (pact.FormFieldDecoded, error) {
text, _ := in.Value.(string)
if text == "bad" {
return pact.FormFieldDecoded{}, &pact.FormFieldError{Messages: []string{"bad in " + in.Locale}}
}
out := pact.FormFieldDecoded{Value: strings.TrimSpace(text)}
if out.Value != text {
out.Notices = []pact.FormFieldNotice{{Message: "trimmed " + in.Locale}}
}
return out, nil
},
ListValue: func(_ context.Context, value any) string { return fmt.Sprintf("list:%v", value) },
}
}
const ftBodyField = " body:\n type: notesml\n label: Body\n"
// ftMLPlugin has both types, a translatable body field and typed columns.
func ftMLPlugin() ftPlugin {
fsys := ftFS(ftNotesField, ftBodyField)
fsys["models/note/columns.yaml"] = &fstest.MapFile{Data: []byte("columns:\n title:\n label: Title\n notes:\n label: Notes\n type: notes\n body:\n label: Body\n type: notesml\n")}
return ftPlugin{id: "acme.demo", fsys: fsys, types: []pact.FormFieldType{ftNotesType(), ftNotesMLType()}}
}
// ftHookController records FieldWritesFromContext in its update hooks.
type ftHookController struct {
ftController
before, after *map[string][]FieldWrite
}
func (c ftHookController) FormBeforeUpdate(ctx context.Context, _ any) error {
writes, ok := FieldWritesFromContext(ctx)
if !ok {
return errors.New("no field writes in FormBeforeUpdate")
}
*c.before = writes
return nil
}
func (c ftHookController) FormAfterUpdate(ctx context.Context, _ any) error {
writes, ok := FieldWritesFromContext(ctx)
if !ok {
return errors.New("no field writes in FormAfterUpdate")
}
*c.after = writes
return nil
}
// ftSave runs the decode half of a save the way CRUDService.save does: the
// ML lift, the plugin lift, the projection and decodePluginFields.
func ftSave(t *testing.T, cc *CompiledController, writer TranslationWriter, target any, body map[string]any, update bool) (pluginDecodeResult, map[string]any, []mlValue, error) {
t.Helper()
op := "create"
if update {
op = "update"
}
translations, err := liftMLValues(context.Background(), cc, body, op, writer, nil)
if err != nil {
t.Fatalf("lift: %v", err)
}
values := liftPluginValues(cc, body, op)
projected := projectOperation(cc, body, op)
res, err := decodePluginFields(context.Background(), pluginDecodeInput{
Writer: writer, CC: cc, Target: target, Values: values, Projected: projected, Translations: translations, Update: update,
})
return res, projected, translations, err
}
func ftWriter() *recordingWriter {
return &recordingWriter{defaultLocale: "pl", enabled: []string{"pl", "en"}}
}
// ftUpload posts one multipart part to the field upload route.
func ftUpload(t *testing.T, svc *service, field, filename string, content []byte, principal *bouncer.Principal, ajax bool) *httptest.ResponseRecorder {
t.Helper()
var buf bytes.Buffer
mw := multipart.NewWriter(&buf)
part, err := mw.CreateFormFile("file", filename)
if err != nil {
t.Fatal(err)
}
if _, err := part.Write(content); err != nil {
t.Fatal(err)
}
if err := mw.Close(); err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodPost, adminAPI("/acme/demo/notes/0/fields/"+field+"/upload"), &buf)
req.Header.Set("Content-Type", mw.FormDataContentType())
if ajax {
req.Header.Set("X-Requested-With", "XMLHttpRequest")
}
for name, value := range map[string]string{"vendor": "acme", "plugin": "demo", "controller": "notes", "id": "0", "field": field} {
req.SetPathValue(name, value)
}
req = req.WithContext(bouncer.WithUser(req.Context(), principal))
rec := httptest.NewRecorder()
requireAjax(svc.fieldUpload)(rec, req)
return rec
}
func TestFieldTypeExpansionSmoke(t *testing.T) {
reg, err := ftCompile(ftMLPlugin())
if err != nil {
t.Fatalf("compile: %v", err)
}
cc, _ := reg.Get("acme.demo.notes")
t.Run("translatable decode: default on the host, others through the writer", func(t *testing.T) {
writer := ftWriter()
res, projected, translations, err := ftSave(t, cc, writer, &ftNote{ID: 1}, map[string]any{"body": map[string]any{"pl": " P ", "en": " E "}, "notes": "n"}, false)
if err != nil {
t.Fatal(err)
}
if projected["body"] != "P" || projected["notes"] != "n" {
t.Fatalf("projected=%v", projected)
}
if err := applyMLTranslations(context.Background(), nil, writer, &ftNote{ID: 1}, translations); err != nil {
t.Fatal(err)
}
if writer.attrs["en"]["body"] != "E" || writer.attrs["pl"] != nil {
t.Fatalf("translations written=%v", writer.attrs)
}
if len(res.writes["body"]) != 2 || len(res.writes["notes"]) != 1 {
t.Fatalf("writes=%v", res.writes)
}
})
t.Run("notices carry the locale of a translatable value", func(t *testing.T) {
res, _, _, err := ftSave(t, cc, ftWriter(), &ftNote{ID: 1}, map[string]any{"body": map[string]any{"pl": " P ", "en": " E "}, "notes": " n "}, false)
if err != nil {
t.Fatal(err)
}
got := localizeNotices(context.Background(), nil, res.notices)
want := []FieldNotice{
{Field: "notes", Message: "acme.demo::lang.trimmed"},
{Field: "body", Locale: "pl", Message: "trimmed pl"},
{Field: "body", Locale: "en", Message: "trimmed en"},
}
if fmt.Sprint(got) != fmt.Sprint(want) {
t.Fatalf("notices=%+v, want %+v", got, want)
}
})
t.Run("a FormFieldError on a non-default locale is a 422 with no notices", func(t *testing.T) {
writer := ftWriter()
res, _, _, err := ftSave(t, cc, writer, &ftNote{ID: 1}, map[string]any{"body": map[string]any{"pl": " P ", "en": "bad"}}, false)
var invalid *ValidationError
if !errors.As(err, &invalid) {
t.Fatalf("err=%v", err)
}
if msgs, _ := invalid.Details["body"].([]string); len(msgs) != 1 || msgs[0] != "bad in en" {
t.Fatalf("details=%v", invalid.Details)
}
if len(res.notices) != 0 || len(writer.attrs) != 0 {
t.Fatalf("notices=%v attrs=%v", res.notices, writer.attrs)
}
})
t.Run("untouched values are skipped on update", func(t *testing.T) {
writer := ftWriter()
writer.attrs = map[string]map[string]string{"en": {"body": "E"}}
target := &ftNote{ID: 1, Body: "P", Notes: "stored"}
res, projected, translations, err := ftSave(t, cc, writer, target, map[string]any{"body": map[string]any{"pl": "P", "en": "E"}, "notes": "served:stored"}, true)
if err != nil {
t.Fatal(err)
}
if _, ok := projected["body"]; ok {
t.Fatalf("untouched host value projected: %v", projected)
}
if _, ok := projected["notes"]; ok {
t.Fatalf("untouched plain value projected: %v", projected)
}
writer.attrs = nil
if err := applyMLTranslations(context.Background(), nil, writer, target, translations); err != nil {
t.Fatal(err)
}
if len(writer.ids) != 0 || len(res.notices) != 0 || len(res.writes) != 0 {
t.Fatalf("ids=%v notices=%v writes=%v", writer.ids, res.notices, res.writes)
}
writer.attrs = map[string]map[string]string{"en": {"body": "E"}}
res, projected, translations, err = ftSave(t, cc, writer, target, map[string]any{"body": map[string]any{"pl": "P", "en": " E2"}}, true)
if err != nil {
t.Fatal(err)
}
if _, ok := projected["body"]; ok {
t.Fatalf("untouched host value projected: %v", projected)
}
if len(translations) != 1 || translations[0].values["en"] != "E2" {
t.Fatalf("translations=%v", translations)
}
if got := localizeNotices(context.Background(), nil, res.notices); len(got) != 1 || got[0].Locale != "en" {
t.Fatalf("notices=%+v", got)
}
})
t.Run("nothing is skipped on create", func(t *testing.T) {
writer := ftWriter()
writer.attrs = map[string]map[string]string{"en": {"body": "E"}}
target := &ftNote{ID: 1, Body: "P", Notes: "stored"}
_, projected, translations, err := ftSave(t, cc, writer, target, map[string]any{"body": map[string]any{"pl": "P", "en": "E"}, "notes": "served:stored"}, false)
if err != nil {
t.Fatal(err)
}
if projected["body"] != "P" || projected["notes"] != "served:stored" || translations[0].values["en"] != "E" {
t.Fatalf("projected=%v translations=%v", projected, translations)
}
})
t.Run("Form hooks read the written values", func(t *testing.T) {
if _, ok := FieldWritesFromContext(context.Background()); ok {
t.Fatal("writes outside a save")
}
res, _, _, err := ftSave(t, cc, ftWriter(), &ftNote{ID: 1}, map[string]any{"body": map[string]any{"pl": "P", "en": "E"}, "notes": "n"}, false)
if err != nil {
t.Fatal(err)
}
var before, after map[string][]FieldWrite
hooked := *cc
hooked.Controller = ftHookController{before: &before, after: &after}
ctx := withFieldWrites(context.Background(), res.writes)
if err := formBeforeUpdate(ctx, &hooked, &ftNote{}); err != nil {
t.Fatal(err)
}
if err := formAfterUpdate(ctx, &hooked, &ftNote{}); err != nil {
t.Fatal(err)
}
want := map[string][]FieldWrite{
"body": {{Locale: "pl", Default: true, Value: "P"}, {Locale: "en", Value: "E"}},
"notes": {{Default: true, Value: "n"}},
}
if fmt.Sprint(before) != fmt.Sprint(want) || fmt.Sprint(after) != fmt.Sprint(want) {
t.Fatalf("before=%v after=%v", before, after)
}
before["body"][0].Value = "changed"
again, _ := FieldWritesFromContext(ctx)
if again["body"][0].Value != "P" {
t.Fatal("FieldWritesFromContext does not return a copy")
}
exported := WithFieldWrites(context.Background(), want)
if got, ok := FieldWritesFromContext(exported); !ok || len(got) != 2 {
t.Fatalf("WithFieldWrites: %v %v", got, ok)
}
})
t.Run("translatable values are loaded per locale", func(t *testing.T) {
data := map[string]any{"body": map[string]string{"pl": "P", "en": "E"}, "notes": "stored"}
if err := hydratePluginFields(context.Background(), nil, cc, ftWriter(), &ftNote{}, data, "update"); err != nil {
t.Fatal(err)
}
if fmt.Sprint(data["body"]) != fmt.Sprint(map[string]string{"pl": "P", "en": "E"}) || data["notes"] != "served:stored" {
t.Fatalf("data=%v", data)
}
})
t.Run("a list column of a plugin type shows ListValue", func(t *testing.T) {
if cc.List == nil {
t.Fatal("no list")
}
row := projectRowCtx(context.Background(), &ftNote{ID: 3, Title: "T", Notes: "raw", Body: "x"}, cc.Controller, cc.List.Columns)
if row["body"] != "list:x" || row["notes"] != "raw" || row["title"] != "T" {
t.Fatalf("row=%v", row)
}
raw, _ := json.Marshal(cc.List.Columns)
if !strings.Contains(string(raw), `"type":"notesml"`) {
t.Fatalf("columns=%s", raw)
}
})
db, _ := openDryRunDB(t)
app := backpack.New(nil)
if err := app.Publish(db); err != nil {
t.Fatal(err)
}
svc := &service{app: app, reg: reg, uploadBytes: 4096}
admin := &bouncer.Principal{ID: 1, Backend: true, PermissionGrants: map[string]bool{"acme.demo.access": true}}
t.Run("upload answers 201 with the type's URL", func(t *testing.T) {
rec := ftUpload(t, svc, "notes", "a.txt", []byte("hello"), admin, true)
if rec.Code != http.StatusCreated {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
var body struct {
Data FieldUploadResponse `json:"data"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
if body.Data.URL != "/media/a.txt?bytes=5&record=nil&rows=4" {
t.Fatalf("url=%q", body.Data.URL)
}
})
t.Run("upload to a core field or a type without Upload is 404", func(t *testing.T) {
for _, field := range []string{"title", "body", "missing"} {
if rec := ftUpload(t, svc, field, "a.txt", []byte("x"), admin, true); rec.Code != http.StatusNotFound {
t.Fatalf("%s: status=%d body=%s", field, rec.Code, rec.Body.String())
}
}
})
t.Run("upload without the controller permission is 403", func(t *testing.T) {
other := &bouncer.Principal{ID: 2, Backend: true}
if rec := ftUpload(t, svc, "notes", "a.txt", []byte("x"), other, true); rec.Code != http.StatusForbidden {
t.Fatalf("status=%d", rec.Code)
}
})
t.Run("upload without X-Requested-With is refused", func(t *testing.T) {
if rec := ftUpload(t, svc, "notes", "a.txt", []byte("x"), admin, false); rec.Code != http.StatusForbidden {
t.Fatalf("status=%d", rec.Code)
}
})
t.Run("upload over the cap is 413", func(t *testing.T) {
if rec := ftUpload(t, svc, "notes", "a.txt", bytes.Repeat([]byte("x"), 8192), admin, true); rec.Code != http.StatusRequestEntityTooLarge {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
})
t.Run("upload FormFieldError is 422 on the field", func(t *testing.T) {
rec := ftUpload(t, svc, "notes", "bad.txt", []byte("x"), admin, true)
if rec.Code != http.StatusUnprocessableEntity || !strings.Contains(rec.Body.String(), `"notes":["acme.demo::lang.bad_file"]`) {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
})
}

View File

@@ -0,0 +1,159 @@
package cabana
import (
"context"
"errors"
"io"
"net/http"
"strings"
"git.golem15.com/golem15/summercms/modules/lagoon"
"git.golem15.com/golem15/summercms/modules/pact"
"gorm.io/gorm"
)
// fieldUpload serves POST .../{id}/fields/{field}/upload (D-18): one file
// an element of a plugin field type uploads, stored by the type's Upload.
// cabana owns the route: the backend guard and the X-Requested-With check
// (mounted with requireAjax), the controller's permissions (protect), the
// field and its context, the record scope and the body cap. {id} 0 is the
// create form and passes no record; any other id loads the record through
// the controller's FormExtendQuery scope inside a transaction, and a record
// outside it is 404. The body is one multipart part named file. It answers
// 201 {data: {url}}; a pact.FormFieldError is a 422 on the field, any other
// Upload error the opaque 500.
func (s *service) fieldUpload(w http.ResponseWriter, r *http.Request) {
s.protect(w, r, func(cc *CompiledController) {
id, err := pathID(r)
if err != nil {
writeCRUDError(w, err)
return
}
field, ok := uploadField(cc, r.PathValue("field"), id)
if !ok {
writeNotFound(w, r)
return
}
db, err := s.db()
if err != nil {
WriteError(w, http.StatusInternalServerError, "error", msgServerError)
return
}
body := &bodyReader{r: http.MaxBytesReader(w, r.Body, s.fieldUploadCap())}
r.Body = io.NopCloser(body)
mr, err := r.MultipartReader()
if err != nil {
writeCRUDError(w, uploadInvalid(field.Name))
return
}
part, err := mr.NextPart()
if err != nil {
s.writeFieldUploadError(w, body, field, uploadInvalid(field.Name))
return
}
if part.FormName() != "file" || strings.TrimSpace(part.FileName()) == "" {
writeCRUDError(w, uploadInvalid(field.Name))
return
}
run := func(ctx context.Context, tx *gorm.DB, record any) (pact.FormFieldUploadResult, error) {
out, err := field.fieldType.def.Upload(ctx, pact.FormFieldUpload{
DB: tx,
Record: record,
Field: field.Name,
Options: copyJSONMap(field.TypeOptions),
Filename: part.FileName(),
Body: part,
})
if err != nil {
if tooLarge(body) {
return pact.FormFieldUploadResult{}, body.err
}
return pact.FormFieldUploadResult{}, fieldTypeFailure(ctx, s.translator(), cc, field, "upload", err)
}
// Exactly one part: anything after file is refused.
if _, err := mr.NextPart(); !errors.Is(err, io.EOF) {
return pact.FormFieldUploadResult{}, uploadInvalid(field.Name)
}
return out, nil
}
var result pact.FormFieldUploadResult
if id == 0 {
result, err = run(r.Context(), db, nil)
} else {
err = lagoon.Transaction(r.Context(), db, func(ctx context.Context, tx *gorm.DB) error {
ctx = withTx(ctx, tx)
target, err := newWritableModel(cc)
if err != nil {
return err
}
pk, err := coercePK(target, id)
if err != nil {
return err
}
if err := loadRecord(ctx, tx, cc, target, pk); err != nil {
return err
}
result, err = run(ctx, tx, target)
return err
})
}
if err != nil {
s.writeFieldUploadError(w, body, field, err)
return
}
WriteData(w, http.StatusCreated, FieldUploadResponse{URL: result.URL}, nil)
})
}
// uploadField is the form field the upload route names: a field of the
// controller's form whose plugin type has Upload and whose context allows
// the operation (create for id 0, update otherwise).
func uploadField(cc *CompiledController, name string, id uint) (FormField, bool) {
if cc == nil || cc.Form == nil {
return FormField{}, false
}
op := "update"
if id == 0 {
op = "create"
}
for _, field := range cc.Form.Fields {
if field.Name != name {
continue
}
if field.fieldType == nil || field.fieldType.def.Upload == nil || !contextAllows(cc, name, op) {
return FormField{}, false
}
return field, true
}
return FormField{}, false
}
// fieldUploadCap is the body cap of the field upload route:
// http.body_limits.upload_bytes, or the default upload cap.
func (s *service) fieldUploadCap() int64 {
if s != nil && s.uploadBytes > 0 {
return s.uploadBytes
}
return defaultUploadCap
}
// tooLarge reports whether the body read hit the cap.
func tooLarge(body *bodyReader) bool {
var limit *http.MaxBytesError
return body != nil && errors.As(body.err, &limit)
}
func uploadInvalid(field string) error {
return &ValidationError{Details: fieldDetail(field, "The "+field+" upload is invalid.")}
}
// writeFieldUploadError maps an upload failure: a body past the cap is 413,
// everything else goes through writeCRUDError.
func (s *service) writeFieldUploadError(w http.ResponseWriter, body *bodyReader, field FormField, err error) {
var limit *http.MaxBytesError
if tooLarge(body) || errors.As(err, &limit) {
WriteError(w, http.StatusRequestEntityTooLarge, "payload_too_large", msgPayloadTooLarge)
return
}
writeCRUDError(w, err)
}

View File

@@ -355,6 +355,10 @@ func (s *service) mount(r pact.Router) {
constrainFileID(g)
g.Get("/{vendor}/{plugin}/{controller}/{id}/files/{field}/{file}/thumb", s.fileThumb)
constrainFileID(g)
// Uploads of plugin field type elements (D-18): cabana owns the route,
// the type's Upload only stores the file. {id} 0 is the create form.
g.Post("/{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload", requireAjax(s.fieldUpload))
constrainFile(g)
})
// The SPA shell: public, no guard. ServeMux prefers every API pattern
// above over the {path...} wildcard.
@@ -1074,6 +1078,13 @@ func (s *service) operationDeclared(w http.ResponseWriter, r *http.Request, cc *
}
func projectRow(row any, controller pact.AdminController, cols []ListColumn) map[string]any {
return projectRowCtx(context.Background(), row, controller, cols)
}
// projectRowCtx is one list row: the id and every visible column. A column
// whose type is a plugin field type with ListValue shows the type's text for
// the stored value.
func projectRowCtx(ctx context.Context, row any, controller pact.AdminController, cols []ListColumn) map[string]any {
v := reflect.ValueOf(row)
for v.Kind() == reflect.Pointer {
if v.IsNil() {
@@ -1100,6 +1111,10 @@ func projectRow(row any, controller pact.AdminController, cols []ListColumn) map
if !field.IsValid() || !field.CanInterface() {
continue
}
if ft := col.fieldType; ft != nil && ft.def.ListValue != nil {
out[col.Key] = ft.def.ListValue(ctx, derefValue(field.Interface()))
continue
}
out[col.Key] = field.Interface()
}
return out

View File

@@ -81,6 +81,13 @@ func CompileList(pluginID string, ctl pact.AdminController, fsys fs.FS) (*ListSc
}
func compileList(pluginID string, ctl pact.AdminController, fsys fs.FS) (*ListSchema, error) {
return compileListWithTypes(pluginID, ctl, fsys, nil)
}
// compileListWithTypes is compileList with the plugin field types of the
// boot: a column may name a plugin alias as its type, and the list then
// shows the type's ListValue for it.
func compileListWithTypes(pluginID string, ctl pact.AdminController, fsys fs.FS, types *fieldTypeRegistry) (*ListSchema, error) {
if ctl == nil {
return nil, fmt.Errorf("cabana: admin schema %s: controller is nil", pluginID)
}
@@ -107,7 +114,7 @@ func compileList(pluginID string, ctl pact.AdminController, fsys fs.FS) (*ListSc
if err != nil {
return nil, bootErr(pluginID, ctl.ID(), cfgPath, err)
}
columns, err := compileColumns(pluginID, ctl, fsys, colPath)
columns, err := compileColumns(pluginID, ctl, fsys, colPath, types)
if err != nil {
return nil, err
}
@@ -197,7 +204,7 @@ func compileList(pluginID string, ctl pact.AdminController, fsys fs.FS) (*ListSc
return schema, nil
}
func compileColumns(pluginID string, ctl pact.AdminController, fsys fs.FS, colPath string) ([]ListColumn, error) {
func compileColumns(pluginID string, ctl pact.AdminController, fsys fs.FS, colPath string, types *fieldTypeRegistry) ([]ListColumn, error) {
colRaw, err := readAsset(fsys, colPath)
if err != nil {
return nil, bootErr(pluginID, ctl.ID(), colPath, err)
@@ -232,9 +239,15 @@ func compileColumns(pluginID string, ctl pact.AdminController, fsys fs.FS, colPa
typ = inferred
}
}
var pluginType *compiledFieldType
if typ != "" {
if _, known := listColumnTypes[typ]; !known {
return nil, bootErr(pluginID, ctl.ID(), colPath, fmt.Errorf("unsupported column type %s", typ))
if types != nil && !types.refuse {
pluginType = types.byName[typ]
}
if pluginType == nil {
return nil, bootErr(pluginID, ctl.ID(), colPath, fmt.Errorf("unsupported column type %s", typ))
}
}
}
if spec.Relation != "" || spec.Select != "" {
@@ -267,6 +280,7 @@ func compileColumns(pluginID string, ctl pact.AdminController, fsys fs.FS, colPa
Relation: spec.Relation,
Select: spec.Select,
Invisible: spec.Invisible,
fieldType: pluginType,
})
}
return compiled, nil

View File

@@ -103,6 +103,7 @@ func TestPhase10Coverage(t *testing.T) {
"POST /{vendor}/{plugin}/{controller}/toolbar/{action}",
"POST /{vendor}/{plugin}/{controller}/widgets/{field}",
"POST /{vendor}/{plugin}/{controller}/{id}/actions/{action}",
"POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload",
"POST /{vendor}/{plugin}/{controller}/{id}/files/{field}",
"POST /{vendor}/{plugin}/{controller}/{id}/files/{field}/reorder",
"POST /{vendor}/{plugin}/{controller}/{id}/relations/{name}/delete",
@@ -120,7 +121,7 @@ func TestPhase10Coverage(t *testing.T) {
"PUT /{vendor}/{plugin}/{controller}/{id}/relations/{name}/records/{child}/files/{field}/{file}",
}
if strings.Join(unsafe, "\n") != strings.Join(want, "\n") {
t.Fatalf("unsafe routes changed; extend TestPhase10CSRF (it expects 29 besides login):\n%s", strings.Join(unsafe, "\n"))
t.Fatalf("unsafe routes changed; extend TestPhase10CSRF (it expects 30 besides login):\n%s", strings.Join(unsafe, "\n"))
}
// The routes added in Phase 10 are safe reads: GET /lang and the shared
// nested pattern serving field options, filter options and relation lists.

View File

@@ -71,9 +71,9 @@ func TestPhase10CSRF(t *testing.T) {
// unlink, file
// upload, file reorder, file caption, file remove, relation child
// create, update and delete, pivot update, child file upload, reorder,
// caption and remove
if unsafe != 29 {
t.Fatalf("walked %d state-changing routes, want 29: %v", unsafe, router.order)
// caption and remove, plugin field type upload
if unsafe != 30 {
t.Fatalf("walked %d state-changing routes, want 30: %v", unsafe, router.order)
}
loginHandler := router.handlers[login]

View File

@@ -153,7 +153,7 @@ func ExecuteList(ctx context.Context, db *gorm.DB, cc *CompiledController, in Li
for i := 0; i < values.Len(); i++ {
record := values.Index(i).Addr().Interface()
records = append(records, record)
data = append(data, projectRow(record, cc.Controller, cc.List.Columns))
data = append(data, projectRowCtx(ctx, record, cc.Controller, cc.List.Columns))
}
states, err := listRowStates(ctx, db, cc, records)
if err != nil {

View File

@@ -79,7 +79,7 @@ func compileRegistryWithTypes(items []controllerRef, types *fieldTypeRegistry) (
if !ok || fsys == nil || fsys.AdminFS() == nil {
return nil, fmt.Errorf("cabana: plugin %s has admin controllers but no AdminFS", item.plugin.ID())
}
list, err := compileList(item.plugin.ID(), item.ctl, fsys.AdminFS())
list, err := compileListWithTypes(item.plugin.ID(), item.ctl, fsys.AdminFS(), types)
if err != nil {
return nil, err
}

View File

@@ -26,6 +26,10 @@ type ListColumn struct {
// not shown: the admin renders no header or cell for it and list rows do
// not carry its value (columns.yaml invisible).
Invisible bool `json:"invisible,omitempty"`
// fieldType is the plugin field type a column names as its type; the
// list then shows the type's ListValue.
fieldType *compiledFieldType
}
// ListFilter is one compiled config_filter scope. Values stay typed scalars.

View File

@@ -89,6 +89,7 @@ var phase09Routes = []adminRoute{
{key: "DELETE /{vendor}/{plugin}/{controller}/{id}/files/{field}/{file}"},
{key: "GET /{vendor}/{plugin}/{controller}/{id}/files/{field}/{file}/download"},
{key: "GET /{vendor}/{plugin}/{controller}/{id}/files/{field}/{file}/thumb"},
{key: "POST /{vendor}/{plugin}/{controller}/{id}/fields/{field}/upload"},
{key: "GET /assets/{vendor}/{plugin}/{file...}", public: true, spa: true},
{key: "GET ", public: true, spa: true},
{key: "GET /{path...}", public: true, spa: true},
@@ -301,6 +302,7 @@ func phase09ProtectedCalls() []phase09Call {
{"record-action", (*service).recordAction},
{"widget-action", (*service).widgetAction},
{"toolbar-action", (*service).toolbarAction},
{"field-upload", (*service).fieldUpload},
{"partial", (*service).partial},
{"show", (*service).show},
{"update", (*service).update},

View File

@@ -65,3 +65,56 @@ func VirtualFieldsFromContext(ctx context.Context) (map[string]any, bool) {
}
return out, true
}
type fieldWritesKey struct{}
// FieldWrite is one value a save wrote for a plugin field type
// (pact.HasFormFieldTypes): Locale is the content locale of a translatable
// type's value and empty otherwise, Default is true when the value went to
// the host column, and Value is what the type's Decode returned.
type FieldWrite struct {
Locale string
Default bool
Value any
}
// withFieldWrites returns ctx carrying the plugin field values a save writes.
func withFieldWrites(ctx context.Context, writes map[string][]FieldWrite) context.Context {
if writes == nil {
writes = map[string][]FieldWrite{}
}
return context.WithValue(ctx, fieldWritesKey{}, writes)
}
// WithFieldWrites returns ctx carrying writes as a save would, so a test can
// call a Form hook that reads FieldWritesFromContext. The map is copied.
func WithFieldWrites(ctx context.Context, writes map[string][]FieldWrite) context.Context {
return withFieldWrites(ctx, copyFieldWrites(writes))
}
// FieldWritesFromContext returns the values a create or update save
// actually writes for the form's plugin field types, keyed by field name, for
// the context handed to a Form hook (pact.FormBeforeCreate,
// pact.FormAfterCreate, pact.FormBeforeUpdate, pact.FormAfterUpdate). A field
// or locale whose submitted value equals the value the form was served is
// not written and not listed; a translatable type lists one entry per
// written locale. The map is a copy. The second result is false outside a
// create or update save.
func FieldWritesFromContext(ctx context.Context) (map[string][]FieldWrite, bool) {
if ctx == nil {
return nil, false
}
writes, ok := ctx.Value(fieldWritesKey{}).(map[string][]FieldWrite)
if !ok {
return nil, false
}
return copyFieldWrites(writes), true
}
func copyFieldWrites(in map[string][]FieldWrite) map[string][]FieldWrite {
out := make(map[string][]FieldWrite, len(in))
for field, list := range in {
out[field] = append([]FieldWrite(nil), list...)
}
return out
}