diff --git a/.planning/phases/07-admin-forms-lists/07-RESEARCH.md b/.planning/phases/07-admin-forms-lists/07-RESEARCH.md new file mode 100644 index 0000000..6408a2d --- /dev/null +++ b/.planning/phases/07-admin-forms-lists/07-RESEARCH.md @@ -0,0 +1,602 @@ +# Phase 7: Admin Forms & Lists - Research + +**Researched:** 2026-02-05 +**Domain:** YAML-driven form and list generation for admin backend +**Confidence:** HIGH (based on WinterCMS reference + official library documentation) + +## Summary + +This research covers the implementation of YAML-driven form and list generation for SummerCMS admin backend. The WinterCMS reference implementation provides a comprehensive blueprint with `fields.yaml` for forms and `columns.yaml` for lists, supported by a widget system for field rendering. + +The standard approach involves: +1. YAML parsing into typed Scala case classes representing form fields and list columns +2. A widget registry system for mapping field types to rendering implementations +3. ScalaTags-based HTML generation with HTMX for interactivity +4. Type-safe validation defined in Scala models (not YAML) + +**Primary recommendation:** Use circe-yaml for YAML parsing into strongly-typed ADTs, ScalaTags for HTML generation, and HTMX for AJAX-like form submission and partial updates. + +## Standard Stack + +### Core +| Library | Version | Purpose | Why Standard | +|---------|---------|---------|--------------| +| circe-yaml | 1.x | YAML parsing | Bridges YAML to Circe JSON AST with type-safe ADT marshalling | +| circe | 0.14.x | JSON/YAML codec derivation | Compile-time derivation of encoders/decoders | +| ScalaTags | 0.13.x | HTML generation | Type-safe, composable HTML fragments | +| HTMX | 2.x | Frontend interactivity | Declarative AJAX, form submission, partial updates | + +### Supporting +| Library | Version | Purpose | When to Use | +|---------|---------|---------|-------------| +| zio-config-yaml | 4.x | ZIO-native YAML config | Alternative if deeper ZIO integration needed | +| Sortable.js | 1.15.x | Drag-and-drop reordering | Repeater item reorder, list row reorder | +| Flatpickr | 4.6.x | Date/time picker | DatePicker widget implementation | + +### Alternatives Considered +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| circe-yaml | zio-config-yaml | Better ZIO integration but less flexible for ad-hoc YAML | +| ScalaTags | Pebble templates | Pebble already in stack; ScalaTags better for programmatic generation | +| HTMX | Full SPA (React) | HTMX simpler, server-rendered, aligns with WinterCMS approach | + +**Mill dependencies:** +```scala +ivy"io.circe::circe-yaml:1.0.0", +ivy"io.circe::circe-generic:0.14.9", +ivy"com.lihaoyi::scalatags:0.13.1" +``` + +## Architecture Patterns + +### Recommended Project Structure +``` +summercms/ +├── admin/ +│ ├── forms/ +│ │ ├── Form.scala # Form widget container +│ │ ├── FormField.scala # Field definition ADT +│ │ ├── FormTabs.scala # Tab grouping +│ │ ├── widgets/ # Widget implementations +│ │ │ ├── TextWidget.scala +│ │ │ ├── TextareaWidget.scala +│ │ │ ├── DropdownWidget.scala +│ │ │ ├── CheckboxWidget.scala +│ │ │ ├── DatePickerWidget.scala +│ │ │ ├── RepeaterWidget.scala +│ │ │ └── RelationWidget.scala +│ │ └── WidgetRegistry.scala # Widget type -> renderer mapping +│ ├── lists/ +│ │ ├── ListView.scala # List widget container +│ │ ├── ListColumn.scala # Column definition ADT +│ │ ├── columns/ # Column type implementations +│ │ │ ├── TextColumn.scala +│ │ │ ├── DateColumn.scala +│ │ │ ├── RelationColumn.scala +│ │ │ └── SwitchColumn.scala +│ │ ├── Filter.scala # Filter scopes +│ │ └── Pagination.scala # Pagination handling +│ └── yaml/ +│ ├── FieldsYamlParser.scala # fields.yaml parser +│ ├── ColumnsYamlParser.scala # columns.yaml parser +│ └── YamlSchema.scala # Shared YAML structures +``` + +### Pattern 1: YAML to ADT Parsing +**What:** Parse YAML configuration into strongly-typed Scala ADTs using circe-yaml +**When to use:** Loading fields.yaml and columns.yaml definitions +**Example:** +```scala +// Source: Circe-YAML documentation +import io.circe.yaml.parser +import io.circe.generic.auto._ + +sealed trait FieldType +case object Text extends FieldType +case object Textarea extends FieldType +case object Dropdown extends FieldType +case object Checkbox extends FieldType +case object DatePicker extends FieldType +case object Repeater extends FieldType +case object Relation extends FieldType + +case class TriggerConfig( + action: String, // "show", "hide", "enable", "disable" + field: String, // field name to watch + condition: String // "checked", "unchecked", "value[x]" +) + +case class FormField( + `type`: Option[String] = Some("text"), + label: Option[String] = None, + span: Option[String] = Some("full"), // "left", "right", "full", "auto" + tab: Option[String] = None, + placeholder: Option[String] = None, + comment: Option[String] = None, + commentAbove: Option[String] = None, + default: Option[String] = None, + required: Option[Boolean] = None, + disabled: Option[Boolean] = None, + hidden: Option[Boolean] = None, + context: Option[List[String]] = None, // ["create", "update"] + trigger: Option[TriggerConfig] = None, + options: Option[Map[String, String]] = None, + cssClass: Option[String] = None, + // Repeater-specific + form: Option[FormDefinition] = None, + minItems: Option[Int] = None, + maxItems: Option[Int] = None, + // Relation-specific + select: Option[String] = None, + nameFrom: Option[String] = None, + emptyOption: Option[String] = None +) + +case class FormDefinition( + fields: Map[String, FormField], + tabs: Option[TabsConfig] = None, + secondaryTabs: Option[TabsConfig] = None +) + +// Parsing +val yaml = """ +fields: + title: + label: Title + span: left + content: + type: textarea + tab: Content +""" + +parser.parse(yaml).flatMap(_.as[FormDefinition]) +``` + +### Pattern 2: Widget Registry Pattern +**What:** Map field type strings to widget rendering implementations +**When to use:** Dynamically rendering form fields based on YAML type +**Example:** +```scala +trait FormWidget { + def render(field: FormField, value: Any, context: FormContext): Frag + def getSaveValue(field: FormField, postData: Map[String, Any]): Any +} + +class WidgetRegistry { + private val widgets = mutable.Map[String, FormWidget]() + + def register(typeName: String, widget: FormWidget): Unit = + widgets(typeName) = widget + + def resolve(typeName: String): Option[FormWidget] = + widgets.get(typeName) +} + +// Registration (in plugin boot) +registry.register("text", TextWidget) +registry.register("textarea", TextareaWidget) +registry.register("dropdown", DropdownWidget) +registry.register("checkbox", CheckboxWidget) +registry.register("datepicker", DatePickerWidget) +registry.register("repeater", RepeaterWidget) +registry.register("relation", RelationWidget) +``` + +### Pattern 3: ScalaTags Form Generation +**What:** Generate HTML forms from parsed YAML using ScalaTags +**When to use:** Rendering forms and lists in admin controllers +**Example:** +```scala +// Source: ScalaTags documentation +import scalatags.Text.all._ + +def renderFormField(field: FormField, name: String, value: Any): Frag = { + val fieldType = field.`type`.getOrElse("text") + val widget = widgetRegistry.resolve(fieldType) + + div(cls := s"form-group span-${field.span.getOrElse("full")}", + field.trigger.map(t => Seq( + data.trigger := s"[name='${t.field}']", + data.triggerAction := t.action, + data.triggerCondition := t.condition + )).getOrElse(Nil), + + field.label.map(l => + label(cls := "form-label", + l, + field.required.filter(identity).map(_ => span(cls := "required", "*")) + ) + ), + + field.commentAbove.map(c => p(cls := "help-block before-field", c)), + + widget.map(_.render(field, value, context)).getOrElse( + input(cls := "form-control", `type` := "text", attr("name") := name) + ), + + field.comment.map(c => p(cls := "help-block", c)) + ) +} +``` + +### Pattern 4: HTMX Form Submission with Validation Display +**What:** Handle form submission and validation error display via HTMX +**When to use:** Save forms with server-side validation, display inline errors +**Example:** +```scala +// Form with HTMX attributes +form( + attr("hx-post") := "/admin/posts/update", + attr("hx-target") := "#form-container", + attr("hx-swap") := "innerHTML", + + // Fields rendered here... + + div(cls := "form-buttons", + button(`type` := "submit", cls := "btn btn-primary", + attr("hx-include") := "[name]", + "Save" + ), + button(`type` := "submit", cls := "btn btn-primary", + attr("hx-post") := "/admin/posts/update?close=true", + "Save and Close" + ) + ) +) + +// Server response on validation error (422 status) +def renderFieldWithError(field: FormField, name: String, value: Any, error: String): Frag = { + div(cls := "form-group has-error", + // ... field rendering ... + p(cls := "error-message", error) + ) +} +``` + +### Pattern 5: List Column Rendering +**What:** Render table columns with type-specific formatting +**When to use:** Admin list views with searchable, sortable columns +**Example:** +```scala +case class ListColumn( + label: String, + `type`: Option[String] = Some("text"), + searchable: Option[Boolean] = None, + sortable: Option[Boolean] = Some(true), + invisible: Option[Boolean] = None, + width: Option[String] = None, + relation: Option[String] = None, + select: Option[String] = None, + format: Option[String] = None // for dates, numbers +) + +def renderColumnValue(column: ListColumn, record: Model): Frag = { + val value = column.relation match { + case Some(rel) => record.getRelation(rel).map(_.getAttribute(column.select.getOrElse("name"))) + case None => record.getAttribute(column.columnName) + } + + column.`type`.getOrElse("text") match { + case "date" => formatDate(value, column.format) + case "timetense" => formatTimeTense(value) // "2 hours ago" + case "switch" => renderSwitch(value) + case "partial" => renderPartial(column.path, record) + case _ => span(value.toString) + } +} +``` + +### Anti-Patterns to Avoid +- **YAML in YAML strings:** Don't nest raw YAML strings; use proper nested structures +- **Validation in YAML:** Keep validation rules in Scala models, not YAML definitions +- **Widget type guessing:** Always require explicit `type` or use clear defaults +- **Inline JavaScript:** Use HTMX data attributes, not inline event handlers +- **Direct DOM manipulation:** Let HTMX handle DOM updates from server responses + +## Don't Hand-Roll + +Problems that look simple but have existing solutions: + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| YAML parsing | Custom parser | circe-yaml | Edge cases: anchors, aliases, multi-doc, escaping | +| Date/time picker | Custom calendar | Flatpickr | i18n, timezones, range selection | +| Drag-and-drop | Custom DnD | Sortable.js | Touch support, animations, nested lists | +| Form validation display | Custom error handling | HTMX + 422 status | Standard pattern, proper error semantics | +| Rich text editing | Custom editor | Existing RichEditor | Toolbar, formatting, sanitization | +| Color picker | Custom picker | Existing ColorPicker | Presets, formats, accessibility | + +**Key insight:** Form widgets are deceptively complex. Each widget has edge cases (empty states, validation, disabled mode, preview mode, nested forms). Reuse WinterCMS widget patterns rather than reinventing. + +## Common Pitfalls + +### Pitfall 1: YAML Anchor/Alias Complexity +**What goes wrong:** YAML anchors (&) and aliases (*) create unexpected merge behavior +**Why it happens:** Users expect intuitive override behavior but YAML merge has specific semantics +**How to avoid:** Document merge behavior clearly; test anchor resolution; provide clear error messages for circular references +**Warning signs:** Fields appearing with unexpected values; missing overrides + +### Pitfall 2: Trigger Condition Evaluation +**What goes wrong:** Conditional field visibility (`trigger`) doesn't work as expected +**Why it happens:** Complex condition expressions, field name resolution in nested forms, timing of evaluation +**How to avoid:** +- Use simple condition types: `checked`, `unchecked`, `value[x]`, `empty`, `filled` +- Resolve field names relative to form context (handle repeaters specially) +- Trigger on client-side with JavaScript, not server round-trips +**Warning signs:** Fields that don't show/hide correctly; triggers that work in forms but not in repeaters + +### Pitfall 3: Repeater State Management +**What goes wrong:** Repeater items lose state, duplicate incorrectly, or save wrong data +**Why it happens:** Index-based naming collisions; AJAX add/remove doesn't sync with form state +**How to avoid:** +- Use unique item keys (UUIDs) not sequential indexes +- Send full repeater state on form submit, not diffs +- Re-render entire repeater on add/remove via HTMX partial +**Warning signs:** "Add item" duplicates existing item's data; deleted items reappear + +### Pitfall 4: Relation Widget Complexity +**What goes wrong:** RelationController-style modals don't load, save, or dismiss correctly +**Why it happens:** Modal forms need separate form context, deferred bindings, session keys +**How to avoid:** +- Each modal gets its own form session key +- Use deferred bindings for unsaved parent models +- Clear modal content on close to prevent stale data +**Warning signs:** Related records not saving; modal data persisting across opens + +### Pitfall 5: List Column Type Coercion +**What goes wrong:** Column values display incorrectly (dates as timestamps, booleans as 0/1) +**Why it happens:** No type coercion from database values to display values +**How to avoid:** +- Always apply column type formatting before display +- Handle null values explicitly for each column type +- Test with actual database values, not mocked data +**Warning signs:** Raw database values appearing in list; "null" or "None" displayed + +## Code Examples + +### Complete fields.yaml Parsing +```scala +// Source: WinterCMS fields.yaml structure analysis +import io.circe._ +import io.circe.yaml.parser +import io.circe.generic.semiauto._ + +case class PresetConfig(field: String, `type`: String) + +case class TriggerConfig( + action: String, + field: String, + condition: String +) + +case class FormFieldConfig( + `type`: Option[String] = None, + label: Option[String] = None, + span: Option[String] = None, + size: Option[String] = None, + tab: Option[String] = None, + placeholder: Option[String] = None, + comment: Option[String] = None, + commentAbove: Option[String] = None, + default: Option[Json] = None, + required: Option[Boolean] = None, + disabled: Option[Boolean] = None, + hidden: Option[Boolean] = None, + readOnly: Option[Boolean] = None, + context: Option[Json] = None, // String or List[String] + trigger: Option[TriggerConfig] = None, + preset: Option[PresetConfig] = None, + dependsOn: Option[List[String]] = None, + options: Option[Json] = None, // Map or method name string + cssClass: Option[String] = None, + attributes: Option[Map[String, String]] = None, + // Widget-specific + form: Option[NestedFormConfig] = None, + path: Option[String] = None, + mode: Option[String] = None, + // Repeater + prompt: Option[String] = None, + titleFrom: Option[String] = None, + minItems: Option[Int] = None, + maxItems: Option[Int] = None, + sortable: Option[Boolean] = None, + // Relation + select: Option[String] = None, + nameFrom: Option[String] = None, + emptyOption: Option[String] = None, + // FileUpload + imageWidth: Option[Int] = None, + imageHeight: Option[Int] = None +) + +case class TabsConfig( + fields: Map[String, FormFieldConfig], + stretch: Option[Boolean] = None, + cssClass: Option[String] = None, + paneCssClass: Option[Map[String, String]] = None, + `lazy`: Option[List[String]] = None, + icons: Option[Map[String, String]] = None +) + +case class NestedFormConfig( + fields: Map[String, FormFieldConfig] +) + +case class FieldsYamlConfig( + fields: Option[Map[String, FormFieldConfig]] = None, + tabs: Option[TabsConfig] = None, + secondaryTabs: Option[TabsConfig] = None +) + +object FieldsYamlConfig { + implicit val presetDecoder: Decoder[PresetConfig] = deriveDecoder + implicit val triggerDecoder: Decoder[TriggerConfig] = deriveDecoder + implicit val nestedFormDecoder: Decoder[NestedFormConfig] = deriveDecoder + implicit val fieldDecoder: Decoder[FormFieldConfig] = deriveDecoder + implicit val tabsDecoder: Decoder[TabsConfig] = deriveDecoder + implicit val configDecoder: Decoder[FieldsYamlConfig] = deriveDecoder + + def parse(yaml: String): Either[Error, FieldsYamlConfig] = + parser.parse(yaml).flatMap(_.as[FieldsYamlConfig]) +} +``` + +### Text Widget Implementation +```scala +import scalatags.Text.all._ + +object TextWidget extends FormWidget { + def render(field: FormFieldConfig, name: String, value: Any, ctx: FormContext): Frag = { + val inputType = field.`type`.getOrElse("text") match { + case "password" => "password" + case "email" => "email" + case "url" => "url" + case "number" => "number" + case _ => "text" + } + + input( + cls := "form-control", + `type` := inputType, + attr("name") := name, + attr("value") := Option(value).map(_.toString).getOrElse(""), + field.placeholder.map(p => attr("placeholder") := p), + field.required.filter(identity).map(_ => attr("required") := "required"), + field.disabled.filter(identity).map(_ => attr("disabled") := "disabled"), + field.readOnly.filter(identity).map(_ => attr("readonly") := "readonly"), + field.attributes.map(_.toSeq.map { case (k, v) => attr(k) := v }).getOrElse(Nil) + ) + } + + def getSaveValue(field: FormFieldConfig, postData: Map[String, Any]): Any = + postData.get(field.name).map(_.toString.trim).filter(_.nonEmpty) +} +``` + +### List View with HTMX Sorting and Pagination +```scala +def renderListView(config: ColumnsYamlConfig, records: Seq[Model], pagination: PaginationState): Frag = { + div(cls := "list-widget", + // Toolbar + div(cls := "list-toolbar", + button(cls := "btn btn-primary", + attr("hx-get") := "/admin/posts/create", + attr("hx-target") := "#form-modal .modal-content", + "Create" + ), + button(cls := "btn btn-danger", + attr("hx-post") := "/admin/posts/bulk-delete", + attr("hx-include") := "[name='checked[]']", + attr("hx-confirm") := "Delete selected records?", + "Delete Selected" + ) + ), + + // Table + table(cls := "table list-table", + thead( + tr( + th(cls := "list-checkbox", + input(`type` := "checkbox", cls := "select-all") + ), + config.columns.filter(!_.invisible.getOrElse(false)).map { case (name, col) => + th( + col.sortable.getOrElse(true).option( + attr("hx-get") := s"/admin/posts?sort=$name&dir=${toggleDir(pagination.sortDir)}", + attr("hx-target") := ".list-widget", + cls := "sortable" + ), + col.label, + (pagination.sortColumn == name).option( + i(cls := s"sort-icon ${pagination.sortDir}") + ) + ) + } + ) + ), + tbody( + records.map { record => + tr( + attr("hx-get") := s"/admin/posts/${record.id}/update", + attr("hx-target") := "#form-container", + td(cls := "list-checkbox", + input(`type` := "checkbox", attr("name") := "checked[]", attr("value") := record.id) + ), + config.columns.filter(!_.invisible.getOrElse(false)).map { case (name, col) => + td(renderColumnValue(col, name, record)) + } + ) + } + ) + ), + + // Pagination + renderPagination(pagination) + ) +} +``` + +## State of the Art + +| Old Approach | Current Approach | When Changed | Impact | +|--------------|------------------|--------------|--------| +| jQuery AJAX forms | HTMX declarative | 2020+ | Simpler, less JS, server-rendered | +| PHP template YAML | Type-safe ADT parsing | N/A (new) | Compile-time validation | +| Runtime widget resolution | Registry pattern | Standard | Explicit, testable | +| Inline validation | Submit-only validation | Decision | Simpler UX, server authority | + +**Deprecated/outdated:** +- Bootstrap 3 styles: Use modern CSS Grid/Flexbox +- jQuery plugins: Use vanilla JS or dedicated libraries +- Inline event handlers: Use HTMX data attributes + +## Open Questions + +1. **Lazy tab loading implementation** + - What we know: WinterCMS supports lazy tab loading via `lazy` config + - What's unclear: Best HTMX pattern for lazy content loading in tabs + - Recommendation: Use `hx-trigger="revealed"` on tab content divs + +2. **Custom widget packaging** + - What we know: Plugins register widgets via boot lifecycle + - What's unclear: How to package widget assets (CSS/JS) with plugin + - Recommendation: Define asset bundling convention in plugin system phase + +3. **RelationController modal form context** + - What we know: Modals need separate session keys and form contexts + - What's unclear: How to pass parent context to modal for deferred bindings + - Recommendation: Include session key in modal URL params; store in hidden field + +## Sources + +### Primary (HIGH confidence) +- WinterCMS reference implementation (`golem15-wintercms-starter/modules/backend/`) + - FormField.php - Field definition and trigger/preset handling + - Form.php - Form widget container and field rendering + - Lists.php - List widget with columns and pagination + - ListColumn.php - Column definition and value extraction + - FilterScope.php - Filter scope types and conditions + - Repeater.php - Repeater widget implementation + - RelationController.php - Relation management behavior +- [Circe-YAML GitHub](https://github.com/circe/circe-yaml) - YAML parsing documentation +- [ScalaTags Documentation](https://com-lihaoyi.github.io/scalatags/) - HTML generation + +### Secondary (MEDIUM confidence) +- [HTMX Examples - Inline Validation](https://htmx.org/examples/inline-validation/) - Form validation patterns +- [Handling Form Errors in HTMX](https://dev.to/yawaramin/handling-form-errors-in-htmx-3ncg) - Error display patterns +- [Baeldung - Handling YAML in Scala 3](https://www.baeldung.com/scala/yaml-scala-3) - YAML library comparison + +### Tertiary (LOW confidence) +- WebSearch results on schema-driven form generation patterns +- Community discussions on Scala YAML libraries + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH - Based on official library docs and established patterns +- Architecture: HIGH - Based on thorough WinterCMS reference analysis +- Pitfalls: MEDIUM - Derived from WinterCMS patterns, may have Scala-specific issues +- Code examples: MEDIUM - Adapted from WinterCMS PHP patterns to Scala + +**Research date:** 2026-02-05 +**Valid until:** 2026-03-05 (30 days - stable domain)