summercms-initial-research/.planning/phases/02-plugin-system/02-RESEARCH.md

Phase 2: Plugin System - Research

Researched: 2026-02-05 Domain: Plugin architecture with ZIO, Scala 3 type-safe extensions, YAML manifests, dependency resolution Confidence: HIGH

Summary

This phase establishes the plugin architecture that all SummerCMS features build upon. The research covers plugin discovery, manifest parsing, lifecycle management (register/boot/shutdown), dependency resolution via topological sorting, and type-safe extension APIs.

The recommended approach leverages ZIO's existing patterns: ZLayer for plugin lifecycle and dependency injection, Ref for concurrent-safe plugin state management, Hub for pub/sub event system, and Scope for resource lifecycle. YAML manifest parsing uses circe-yaml (well-maintained, Scala 3 native). Dependency resolution uses either custom Kahn's algorithm or the sciss/topology library. Version constraint handling uses semver-parser-scala for npm-style ranges (^1.2.0). Hot-reload in dev mode uses directory-watcher for file system monitoring.

Primary recommendation: Build plugins as ZIO services with YAML manifests for metadata, Scala traits for lifecycle hooks, Hub-based events for loose coupling, and compile-time type-safe extension traits. Use topological sort for dependency ordering with cycle detection at discovery time.

Standard Stack

The established libraries/tools for this domain:

Core

Library	Version	Purpose	Why Standard
ZIO	2.1.x	Plugin lifecycle, DI	Already in stack, ZLayer pattern fits plugin model
circe-yaml	0.16.1	YAML manifest parsing	Well-maintained, Scala 3 native, circe ecosystem
circe-generic	0.14.x	JSON/YAML derivation	Automatic codec derivation for case classes
semver-parser-scala	0.0.4	Version constraints	Supports ^, ~, range operators (npm-style)

Supporting

Library	Version	Purpose	When to Use
directory-watcher	0.19.1	File watching	Dev mode hot-reload only
sciss/topology	1.1.4	Topological sort	Alternative to hand-rolling dependency resolution

Alternatives Considered

Instead of	Could Use	Tradeoff
circe-yaml	SnakeYAML direct	SnakeYAML requires JavaBeans, less idiomatic Scala
semver-parser-scala	coursier-versions	coursier-versions lacks caret/tilde range syntax
sciss/topology	Hand-roll Kahn's	Library is small, but algorithm is simple enough to implement
directory-watcher	Java WatchService direct	directory-watcher handles macOS native events better

Installation (Mill build.mill):

def mvnDeps = Seq(
  // Existing deps...

  // YAML parsing
  mvn"io.circe::circe-yaml:0.16.1",
  mvn"io.circe::circe-generic:0.14.10",

  // Version constraints
  mvn"com.github.sh4869::semver-parser-scala:0.0.4",

  // File watching (dev mode)
  mvn"io.methvin:directory-watcher:0.19.1"
)

Architecture Patterns

Recommended Plugin Directory Structure

plugins/
├── golem15/
│   ├── blog/
│   │   ├── plugin.yaml           # Manifest (metadata, deps)
│   │   ├── Plugin.scala          # Lifecycle trait implementation
│   │   ├── controllers/          # Backend controllers
│   │   ├── models/               # Domain models
│   │   ├── components/           # Frontend components
│   │   ├── extensions/           # Extensions to other plugins
│   │   ├── resources/
│   │   │   ├── db/migration/     # Plugin-specific migrations
│   │   │   ├── lang/             # Translation files
│   │   │   └── views/            # Templates
│   │   └── routes.scala          # Plugin routes
│   └── user/
│       ├── plugin.yaml
│       └── ...
└── vendor/
    └── external-plugin/
        └── ...

Pattern 1: Plugin Trait with ZIO Lifecycle

What: Define plugins as Scala traits with ZIO effect-based lifecycle hooks When to use: All plugins must implement this trait Example:

// Source: ZIO Service Pattern + CONTEXT.md decisions
trait SummerPlugin:
  /** Plugin identifier (vendor.name) */
  def id: PluginId

  /** Synchronous registration phase - pure data, no effects */
  def register(ctx: PluginContext): PluginRegistration

  /** Async boot phase - can perform effects */
  def boot: ZIO[PluginEnv, PluginError, Unit]

  /** Async shutdown phase - cleanup resources */
  def shutdown: ZIO[Any, Nothing, Unit]

case class PluginRegistration(
  components: List[ComponentDef],
  permissions: List[PermissionDef],
  navigation: List[NavigationDef],
  settings: List[SettingDef],
  events: List[EventSubscription],
  extensions: List[ExtensionDef]
)

Pattern 2: YAML Manifest with Circe Derivation

What: Plugin metadata in YAML, parsed into typed case classes When to use: Every plugin needs a plugin.yaml manifest Example:

# plugins/golem15/blog/plugin.yaml
vendor: golem15
name: blog
version: 1.2.0
description: Blog management plugin
author: SummerCMS Team
homepage: https://summercms.io/plugins/blog
license: MIT

# Dependencies with semver constraints
dependencies:
  golem15.user: "^1.0.0"

# Optional dependencies - load after if present
optionalDependencies:
  golem15.seo: ">=1.0.0"

# Boot priority among plugins at same dependency level (higher = first)
priority: 100

# Plugin can be disabled via admin
enabled: true

// Source: circe-yaml documentation
import io.circe.generic.auto.*
import io.circe.yaml.parser

case class PluginManifest(
  vendor: String,
  name: String,
  version: String,
  description: Option[String],
  author: Option[String],
  homepage: Option[String],
  license: Option[String],
  dependencies: Map[String, String] = Map.empty,
  optionalDependencies: Map[String, String] = Map.empty,
  priority: Int = 0,
  enabled: Boolean = true
):
  def id: PluginId = PluginId(vendor, name)

def parseManifest(yaml: String): Either[io.circe.Error, PluginManifest] =
  parser.parse(yaml).flatMap(_.as[PluginManifest])

Pattern 3: Plugin State Machine with ZIO Ref

What: Track plugin state through lifecycle using atomic Ref When to use: Plugin manager tracks each plugin's state Example:

// Source: ZIO Ref documentation + CONTEXT.md state tracking
enum PluginState:
  case Discovered                    // Found on filesystem
  case Registered                    // register() completed
  case Booting                       // boot() in progress
  case Running                       // boot() succeeded
  case Failed(error: PluginError)    // boot() failed
  case Disabled                      // Manually disabled
  case ShuttingDown                  // shutdown() in progress
  case Stopped                       // shutdown() completed

case class PluginInstance(
  manifest: PluginManifest,
  plugin: SummerPlugin,
  stateRef: Ref[PluginState]
)

class PluginManager(
  plugins: Ref[Map[PluginId, PluginInstance]],
  eventHub: Hub[PluginEvent]
):
  def boot(id: PluginId): ZIO[PluginEnv, PluginError, Unit] =
    for
      instance <- getPlugin(id)
      _        <- instance.stateRef.set(PluginState.Booting)
      result   <- instance.plugin.boot.either
      _        <- result match
        case Right(_) =>
          instance.stateRef.set(PluginState.Running) *>
          eventHub.publish(PluginEvent.Booted(id))
        case Left(err) =>
          instance.stateRef.set(PluginState.Failed(err)) *>
          eventHub.publish(PluginEvent.BootFailed(id, err))
    yield ()

Pattern 4: Dependency Resolution via Topological Sort

What: Resolve plugin load order using Kahn's algorithm on dependency DAG When to use: At discovery time, before any plugins boot Example:

// Source: Topological sort algorithm for dependency resolution
def resolveDependencyOrder(
  plugins: Map[PluginId, PluginManifest]
): Either[CyclicDependencyError, List[PluginId]] =
  // Build adjacency list (plugin -> plugins that depend on it)
  val dependents = plugins.values.foldLeft(Map.empty[PluginId, Set[PluginId]]) {
    (acc, manifest) =>
      manifest.dependencies.keys.foldLeft(acc) { (a, depId) =>
        val parsed = PluginId.parse(depId)
        a.updated(parsed, a.getOrElse(parsed, Set.empty) + manifest.id)
      }
  }

  // Calculate in-degrees
  val inDegree = plugins.keys.map { id =>
    id -> plugins(id).dependencies.size
  }.toMap

  // Kahn's algorithm
  @tailrec
  def process(
    queue: List[PluginId],
    result: List[PluginId],
    degrees: Map[PluginId, Int]
  ): Either[CyclicDependencyError, List[PluginId]] =
    queue match
      case Nil =>
        if result.size == plugins.size then Right(result.reverse)
        else Left(CyclicDependencyError(plugins.keySet -- result.toSet))
      case head :: tail =>
        val newDegrees = dependents.getOrElse(head, Set.empty)
          .foldLeft(degrees) { (d, dep) =>
            d.updated(dep, d(dep) - 1)
          }
        val newReady = dependents.getOrElse(head, Set.empty)
          .filter(d => newDegrees(d) == 0)
          .toList
          .sortBy(id => -plugins(id).priority) // Higher priority first
        process(tail ++ newReady, head :: result, newDegrees)

  val initial = plugins.filter(_._2.dependencies.isEmpty).keys.toList
    .sortBy(id => -plugins(id).priority)
  process(initial, Nil, inDegree)

Pattern 5: Hub-Based Event System

What: Pub/sub events for loose coupling between plugins When to use: Plugin A needs to react to Plugin B's actions without direct dependency Example:

// Source: ZIO Hub documentation + WinterCMS Event pattern
sealed trait SummerEvent
object SummerEvent:
  // Core lifecycle events
  case class PluginBooted(id: PluginId) extends SummerEvent
  case class PluginShutdown(id: PluginId) extends SummerEvent

  // Domain events (plugins define their own)
  trait DomainEvent extends SummerEvent

// Event service
trait EventService:
  def publish[E <: SummerEvent](event: E): UIO[Boolean]
  def subscribe: ZIO[Scope, Nothing, Dequeue[SummerEvent]]
  def subscribeFiltered[E <: SummerEvent: ClassTag]: ZIO[Scope, Nothing, Dequeue[E]]

object EventService:
  val live: ZLayer[Any, Nothing, EventService] =
    ZLayer.scoped {
      Hub.bounded[SummerEvent](256).map { hub =>
        new EventService:
          def publish[E <: SummerEvent](event: E): UIO[Boolean] =
            hub.publish(event)
          def subscribe: ZIO[Scope, Nothing, Dequeue[SummerEvent]] =
            hub.subscribe
          def subscribeFiltered[E <: SummerEvent: ClassTag]: ZIO[Scope, Nothing, Dequeue[E]] =
            hub.subscribe.map(_.filterOutput {
              case e: E => Some(e)
              case _ => None
            })
      }
    }

// Plugin subscribing to events
def boot: ZIO[PluginEnv, PluginError, Unit] =
  ZIO.serviceWithZIO[EventService] { events =>
    events.subscribeFiltered[UserRegistered].flatMap { queue =>
      queue.take.flatMap { event =>
        // Send welcome email when user registers
        sendWelcomeEmail(event.user)
      }.forever.fork
    }
  }

Pattern 6: Type-Safe Extension Traits

What: Plugins declare extension points as traits, other plugins implement them When to use: Plugin A wants to allow Plugin B to extend its functionality Example:

// Source: Scala 3 type classes + CONTEXT.md extension mechanics
// Blog plugin declares extension point
trait BlogExtension:
  def extendPostFields: List[FieldDef]
  def extendPostModel(post: Post): Post
  def validatePost(post: Post): ZIO[Any, ValidationError, Unit]

// SEO plugin implements extension
object SeoExtension extends BlogExtension:
  def extendPostFields: List[FieldDef] = List(
    FieldDef("meta_title", FieldType.Text, tab = "SEO"),
    FieldDef("meta_description", FieldType.Textarea, tab = "SEO"),
    FieldDef("canonical_url", FieldType.Text, tab = "SEO")
  )

  def extendPostModel(post: Post): Post =
    post.copy(extraData = post.extraData ++ Map(
      "meta_title" -> post.title.take(60),
      "meta_description" -> post.excerpt.take(160)
    ))

  def validatePost(post: Post): ZIO[Any, ValidationError, Unit] =
    ZIO.when(post.extraData.get("meta_title").exists(_.length > 60)) {
      ZIO.fail(ValidationError("Meta title must be 60 characters or less"))
    }.unit

// Blog plugin collects all extensions
class BlogPlugin extends SummerPlugin:
  def boot: ZIO[PluginEnv, PluginError, Unit] =
    ZIO.serviceWithZIO[ExtensionRegistry] { registry =>
      registry.getExtensions[BlogExtension].flatMap { extensions =>
        // Merge all field definitions
        val allFields = extensions.flatMap(_.extendPostFields)
        registerFormFields("post", allFields)
      }
    }

Pattern 7: Registration-Based UI Extensions

What: Plugins register navigation, forms, settings via declarative API When to use: Admin UI customization Example:

// Source: WinterCMS pattern adapted to Scala
def register(ctx: PluginContext): PluginRegistration =
  PluginRegistration(
    navigation = List(
      NavigationItem(
        id = "blog",
        label = "lang.blog.menu_label",
        url = "/admin/golem15/blog/posts",
        icon = "pencil",
        permissions = List("golem15.blog.*"),
        order = 300,
        sideMenu = List(
          NavigationItem(
            id = "posts",
            label = "lang.blog.posts",
            url = "/admin/golem15/blog/posts",
            icon = "copy"
          ),
          NavigationItem(
            id = "categories",
            label = "lang.blog.categories",
            url = "/admin/golem15/blog/categories",
            icon = "list-ul"
          )
        )
      )
    ),
    permissions = List(
      Permission("golem15.blog.access_posts", "Access posts"),
      Permission("golem15.blog.manage_settings", "Manage settings")
    ),
    settings = List(
      SettingDef(
        key = "blog",
        label = "Blog Settings",
        modelClass = classOf[BlogSettings],
        icon = "pencil"
      )
    )
  )

Anti-Patterns to Avoid

• Direct plugin-to-plugin imports: Use events or extension traits, not direct class imports
• Synchronous boot effects: Boot must return ZIO, never block
• Mutable shared state: Use Ref for concurrent state, never var
• Ignoring boot failures: Failed plugins must disable dependents, not crash app
• String-based event names: Use sealed trait ADT for events, compile-time safety

Don't Hand-Roll

Problems that look simple but have existing solutions:

Problem	Don't Build	Use Instead	Why
YAML parsing	Manual string parsing	circe-yaml	Edge cases (quotes, escapes, multiline), type safety
Version constraints	String comparison	semver-parser-scala	Caret, tilde, range operators are complex
Topological sort	Naive recursion	Kahn's algorithm or sciss/topology	Cycle detection, efficiency
File watching	Thread.sleep polling	directory-watcher	OS-native events, efficiency, cross-platform
Pub/sub events	Callback lists	ZIO Hub	Back-pressure, concurrency, type safety
Plugin state	Mutable var	ZIO Ref	Atomic updates, concurrent safety

Key insight: Plugin systems seem simple but have many edge cases around ordering, failures, and concurrency. ZIO's primitives (Ref, Hub, ZLayer) solve these elegantly.

Common Pitfalls

Pitfall 1: Circular Dependencies Not Detected Early

What goes wrong: Plugin A requires B, B requires A, discovered at boot time causing crash Why it happens: Dependencies checked lazily during boot instead of at discovery How to avoid: Run topological sort at discovery time, fail fast with clear error message Warning signs: "Plugin X is already booting" errors, stack overflows

Pitfall 2: Partial Boot State

What goes wrong: Plugin boots halfway, crashes, leaves system in inconsistent state Why it happens: Boot effects not atomic, no rollback mechanism How to avoid: Use acquireRelease pattern - register cleanup in reverse order Warning signs: Orphan event listeners, resource leaks, duplicate registrations on retry

Pitfall 3: Extension Ordering Conflicts

What goes wrong: Plugin A and B both extend form field "title", last one wins unexpectedly Why it happens: Extensions merged without conflict detection How to avoid: Detect same-key extensions, error or use explicit priority Warning signs: Missing form fields, unexpected field values

Pitfall 4: Dev Mode Hot-Reload Breaking State

What goes wrong: Plugin reloaded, but old event listeners still active Why it happens: Hot-reload creates new instance without cleanup How to avoid: Full shutdown sequence before reload, track all subscriptions in Scope Warning signs: Duplicate event handling, ghost listeners, memory growth

Pitfall 5: Optional Dependency Load Order

What goes wrong: Plugin loads before its optional dependency, extension not applied Why it happens: Optional deps not in topological sort How to avoid: Include optional deps in sort but tolerate missing Warning signs: Features randomly missing depending on load order

Pitfall 6: Version Constraint Mismatch

What goes wrong: Plugin requires ^1.2.0, installed is 1.1.5, cryptic runtime errors Why it happens: Version constraints not validated at discovery How to avoid: Validate all version constraints before any plugin boots Warning signs: NoSuchMethodError, ClassNotFoundException at runtime

Code Examples

Verified patterns from official sources:

Plugin Discovery Service

// Source: ZIO Service Pattern
import java.nio.file.{Files, Path, Paths}
import scala.jdk.CollectionConverters.*

trait PluginDiscovery:
  def discover(pluginsDir: Path): Task[List[DiscoveredPlugin]]

object PluginDiscovery:
  val live: ZLayer[Any, Nothing, PluginDiscovery] =
    ZLayer.succeed {
      new PluginDiscovery:
        def discover(pluginsDir: Path): Task[List[DiscoveredPlugin]] =
          ZIO.attemptBlocking {
            // Find all plugin.yaml files
            Files.walk(pluginsDir, 3).iterator.asScala
              .filter(_.getFileName.toString == "plugin.yaml")
              .toList
          }.flatMap { manifests =>
            ZIO.foreach(manifests) { path =>
              for
                content  <- ZIO.attemptBlocking(Files.readString(path))
                manifest <- ZIO.fromEither(parseManifest(content))
                  .mapError(e => new Exception(s"Invalid manifest at $path: $e"))
              yield DiscoveredPlugin(path.getParent, manifest)
            }
          }
    }

case class DiscoveredPlugin(
  directory: Path,
  manifest: PluginManifest
)

Version Constraint Validation

// Source: semver-parser-scala documentation
import semver._

def validateDependencies(
  available: Map[PluginId, String], // id -> version
  required: Map[String, String]     // "vendor.name" -> "^1.2.0"
): Either[DependencyError, Unit] =
  required.toList.traverse { case (idStr, constraint) =>
    val id = PluginId.parse(idStr)
    available.get(id) match
      case None =>
        Left(DependencyError.Missing(id))
      case Some(versionStr) =>
        val version = SemVer(versionStr)
        val range = Range(constraint)
        if range.valid(version) then Right(())
        else Left(DependencyError.VersionMismatch(id, constraint, versionStr))
  }.void

Hot-Reload with Directory Watcher

// Source: directory-watcher API
import io.methvin.watcher.{DirectoryWatcher, DirectoryChangeEvent}

trait HotReloadService:
  def watchPlugins: ZIO[Scope, Nothing, Unit]

object HotReloadService:
  def live(pluginsDir: Path): ZLayer[PluginManager, Nothing, HotReloadService] =
    ZLayer.fromFunction { (manager: PluginManager) =>
      new HotReloadService:
        def watchPlugins: ZIO[Scope, Nothing, Unit] =
          ZIO.acquireRelease(
            ZIO.attemptBlocking {
              DirectoryWatcher.builder()
                .path(pluginsDir)
                .listener { event =>
                  // Trigger reload on plugin.yaml or .scala changes
                  if event.path.toString.endsWith(".yaml") ||
                     event.path.toString.endsWith(".scala") then
                    // Extract plugin ID from path
                    val pluginId = extractPluginId(event.path)
                    Runtime.default.unsafe.run(
                      manager.reloadPlugin(pluginId)
                    )
                }
                .build()
            }.orDie
          )(watcher => ZIO.attemptBlocking(watcher.close()).orDie)
          .flatMap(w => ZIO.attemptBlocking(w.watch()).orDie.fork.unit)
    }

Extension Registry

// Source: ZIO Service Pattern + Scala 3 ClassTag
import scala.reflect.ClassTag

trait ExtensionRegistry:
  def register[E: ClassTag](pluginId: PluginId, extension: E): UIO[Unit]
  def getExtensions[E: ClassTag]: UIO[List[E]]
  def getExtension[E: ClassTag](pluginId: PluginId): UIO[Option[E]]

object ExtensionRegistry:
  val live: ZLayer[Any, Nothing, ExtensionRegistry] =
    ZLayer.fromZIO {
      Ref.make(Map.empty[String, Map[PluginId, Any]]).map { registry =>
        new ExtensionRegistry:
          private def key[E: ClassTag]: String =
            summon[ClassTag[E]].runtimeClass.getName

          def register[E: ClassTag](pluginId: PluginId, extension: E): UIO[Unit] =
            registry.update { map =>
              val k = key[E]
              val existing = map.getOrElse(k, Map.empty)
              map.updated(k, existing + (pluginId -> extension))
            }

          def getExtensions[E: ClassTag]: UIO[List[E]] =
            registry.get.map { map =>
              map.getOrElse(key[E], Map.empty)
                .values.toList.asInstanceOf[List[E]]
            }

          def getExtension[E: ClassTag](pluginId: PluginId): UIO[Option[E]] =
            registry.get.map { map =>
              map.getOrElse(key[E], Map.empty)
                .get(pluginId).asInstanceOf[Option[E]]
            }
      }
    }

State of the Art

Old Approach	Current Approach	When Changed	Impact
Runtime reflection for plugins	Compile-time traits + circe derivation	Scala 3	Type safety, better errors
Callback-based events	ZIO Hub pub/sub	ZIO 2.0	Back-pressure, concurrency safety
Mutable plugin state	ZIO Ref	ZIO 2.0	Atomic updates, no race conditions
Manual dependency ordering	Topological sort at discovery	Always best practice	Fail-fast on cycles
Polling for file changes	directory-watcher native events	2020+	Efficiency, responsiveness

Deprecated/outdated:

• Java ServiceLoader: Runtime reflection, no Scala 3 native support, hard to configure
• Akka actors for events: Overkill, ZIO Hub is simpler and sufficient
• YAML via SnakeYAML direct: Requires JavaBeans, less idiomatic for Scala

Open Questions

Things that couldn't be fully resolved:

1. Shutdown Timeout Policy

   • What we know: Plugins need graceful shutdown, but can hang
   • What's unclear: What timeout is appropriate, should it be configurable?
   • Recommendation: Default 30 seconds per plugin, configurable in manifest, force-kill after timeout

2. Event Listener Priority

   • What we know: Multiple plugins may listen to same event
   • What's unclear: Should listeners have ordering? What if one listener should cancel event?
   • Recommendation: Ordered by plugin dependency (dependents first), no cancellation in v1

3. Plugin Isolation Level

   • What we know: Plugins share JVM, no classloader isolation
   • What's unclear: Can a plugin crash the whole app?
   • Recommendation: Use ZIO error handling, defects propagate but are logged, continue running

4. YAML vs Code for Form Extensions

   • What we know: CONTEXT.md says hybrid (simple in YAML, complex in code)
   • What's unclear: Exact boundary, how YAML extensions merge with code
   • Recommendation: Define clear merge rules - YAML first, code can override/extend

Sources

Primary (HIGH confidence)

• ZIO Hub Documentation - Pub/sub pattern
• ZIO Ref Documentation - State management
• ZIO Scope Documentation - Resource lifecycle
• ZIO Architectural Patterns - Service layer design
• circe-yaml GitHub - YAML parsing, version 0.16.1
• directory-watcher GitHub - File watching, version 0.19.1

Secondary (MEDIUM confidence)

• sciss/topology Scaladex - Topological sort library
• semver-parser-scala GitHub - Version constraint parsing
• SoftwareMill ZIO 2 Structure - Service patterns
• Scala 3 Extension Methods - Type-safe extensions

Tertiary (LOW confidence)

• ZIO-NIO Archive - Archived in Jan 2025, patterns still valid
• Topological Sort Guide - Algorithm explanation

Metadata

Confidence breakdown:

• Standard stack: HIGH - All libraries verified via official docs and releases
• Architecture patterns: HIGH - Based on ZIO official documentation and WinterCMS reference
• Pitfalls: MEDIUM - Mix of ZIO docs and WinterCMS experience
• Extension API: MEDIUM - Novel design combining multiple patterns

Research date: 2026-02-05 Valid until: 2026-03-05 (30 days - stable patterns, libraries may update)