# Phase 2: Plugin System - Research **Researched:** 2026-02-05 **Domain:** Plugin architecture with ZIO, Scala 3 type-safe extensions, YAML manifests, dependency resolution **Confidence:** HIGH ## Summary This phase establishes the plugin architecture that all SummerCMS features build upon. The research covers plugin discovery, manifest parsing, lifecycle management (register/boot/shutdown), dependency resolution via topological sorting, and type-safe extension APIs. The recommended approach leverages ZIO's existing patterns: `ZLayer` for plugin lifecycle and dependency injection, `Ref` for concurrent-safe plugin state management, `Hub` for pub/sub event system, and `Scope` for resource lifecycle. YAML manifest parsing uses circe-yaml (well-maintained, Scala 3 native). Dependency resolution uses either custom Kahn's algorithm or the sciss/topology library. Version constraint handling uses semver-parser-scala for npm-style ranges (`^1.2.0`). Hot-reload in dev mode uses directory-watcher for file system monitoring. **Primary recommendation:** Build plugins as ZIO services with YAML manifests for metadata, Scala traits for lifecycle hooks, Hub-based events for loose coupling, and compile-time type-safe extension traits. Use topological sort for dependency ordering with cycle detection at discovery time. ## Standard Stack The established libraries/tools for this domain: ### Core | Library | Version | Purpose | Why Standard | |---------|---------|---------|--------------| | ZIO | 2.1.x | Plugin lifecycle, DI | Already in stack, ZLayer pattern fits plugin model | | circe-yaml | 0.16.1 | YAML manifest parsing | Well-maintained, Scala 3 native, circe ecosystem | | circe-generic | 0.14.x | JSON/YAML derivation | Automatic codec derivation for case classes | | semver-parser-scala | 0.0.4 | Version constraints | Supports ^, ~, range operators (npm-style) | ### Supporting | Library | Version | Purpose | When to Use | |---------|---------|---------|-------------| | directory-watcher | 0.19.1 | File watching | Dev mode hot-reload only | | sciss/topology | 1.1.4 | Topological sort | Alternative to hand-rolling dependency resolution | ### Alternatives Considered | Instead of | Could Use | Tradeoff | |------------|-----------|----------| | circe-yaml | SnakeYAML direct | SnakeYAML requires JavaBeans, less idiomatic Scala | | semver-parser-scala | coursier-versions | coursier-versions lacks caret/tilde range syntax | | sciss/topology | Hand-roll Kahn's | Library is small, but algorithm is simple enough to implement | | directory-watcher | Java WatchService direct | directory-watcher handles macOS native events better | **Installation (Mill build.mill):** ```scala def mvnDeps = Seq( // Existing deps... // YAML parsing mvn"io.circe::circe-yaml:0.16.1", mvn"io.circe::circe-generic:0.14.10", // Version constraints mvn"com.github.sh4869::semver-parser-scala:0.0.4", // File watching (dev mode) mvn"io.methvin:directory-watcher:0.19.1" ) ``` ## Architecture Patterns ### Recommended Plugin Directory Structure ``` plugins/ ├── golem15/ │ ├── blog/ │ │ ├── plugin.yaml # Manifest (metadata, deps) │ │ ├── Plugin.scala # Lifecycle trait implementation │ │ ├── controllers/ # Backend controllers │ │ ├── models/ # Domain models │ │ ├── components/ # Frontend components │ │ ├── extensions/ # Extensions to other plugins │ │ ├── resources/ │ │ │ ├── db/migration/ # Plugin-specific migrations │ │ │ ├── lang/ # Translation files │ │ │ └── views/ # Templates │ │ └── routes.scala # Plugin routes │ └── user/ │ ├── plugin.yaml │ └── ... └── vendor/ └── external-plugin/ └── ... ``` ### Pattern 1: Plugin Trait with ZIO Lifecycle **What:** Define plugins as Scala traits with ZIO effect-based lifecycle hooks **When to use:** All plugins must implement this trait **Example:** ```scala // Source: ZIO Service Pattern + CONTEXT.md decisions trait SummerPlugin: /** Plugin identifier (vendor.name) */ def id: PluginId /** Synchronous registration phase - pure data, no effects */ def register(ctx: PluginContext): PluginRegistration /** Async boot phase - can perform effects */ def boot: ZIO[PluginEnv, PluginError, Unit] /** Async shutdown phase - cleanup resources */ def shutdown: ZIO[Any, Nothing, Unit] case class PluginRegistration( components: List[ComponentDef], permissions: List[PermissionDef], navigation: List[NavigationDef], settings: List[SettingDef], events: List[EventSubscription], extensions: List[ExtensionDef] ) ``` ### Pattern 2: YAML Manifest with Circe Derivation **What:** Plugin metadata in YAML, parsed into typed case classes **When to use:** Every plugin needs a plugin.yaml manifest **Example:** ```yaml # plugins/golem15/blog/plugin.yaml vendor: golem15 name: blog version: 1.2.0 description: Blog management plugin author: SummerCMS Team homepage: https://summercms.io/plugins/blog license: MIT # Dependencies with semver constraints dependencies: golem15.user: "^1.0.0" # Optional dependencies - load after if present optionalDependencies: golem15.seo: ">=1.0.0" # Boot priority among plugins at same dependency level (higher = first) priority: 100 # Plugin can be disabled via admin enabled: true ``` ```scala // Source: circe-yaml documentation import io.circe.generic.auto.* import io.circe.yaml.parser case class PluginManifest( vendor: String, name: String, version: String, description: Option[String], author: Option[String], homepage: Option[String], license: Option[String], dependencies: Map[String, String] = Map.empty, optionalDependencies: Map[String, String] = Map.empty, priority: Int = 0, enabled: Boolean = true ): def id: PluginId = PluginId(vendor, name) def parseManifest(yaml: String): Either[io.circe.Error, PluginManifest] = parser.parse(yaml).flatMap(_.as[PluginManifest]) ``` ### Pattern 3: Plugin State Machine with ZIO Ref **What:** Track plugin state through lifecycle using atomic Ref **When to use:** Plugin manager tracks each plugin's state **Example:** ```scala // Source: ZIO Ref documentation + CONTEXT.md state tracking enum PluginState: case Discovered // Found on filesystem case Registered // register() completed case Booting // boot() in progress case Running // boot() succeeded case Failed(error: PluginError) // boot() failed case Disabled // Manually disabled case ShuttingDown // shutdown() in progress case Stopped // shutdown() completed case class PluginInstance( manifest: PluginManifest, plugin: SummerPlugin, stateRef: Ref[PluginState] ) class PluginManager( plugins: Ref[Map[PluginId, PluginInstance]], eventHub: Hub[PluginEvent] ): def boot(id: PluginId): ZIO[PluginEnv, PluginError, Unit] = for instance <- getPlugin(id) _ <- instance.stateRef.set(PluginState.Booting) result <- instance.plugin.boot.either _ <- result match case Right(_) => instance.stateRef.set(PluginState.Running) *> eventHub.publish(PluginEvent.Booted(id)) case Left(err) => instance.stateRef.set(PluginState.Failed(err)) *> eventHub.publish(PluginEvent.BootFailed(id, err)) yield () ``` ### Pattern 4: Dependency Resolution via Topological Sort **What:** Resolve plugin load order using Kahn's algorithm on dependency DAG **When to use:** At discovery time, before any plugins boot **Example:** ```scala // Source: Topological sort algorithm for dependency resolution def resolveDependencyOrder( plugins: Map[PluginId, PluginManifest] ): Either[CyclicDependencyError, List[PluginId]] = // Build adjacency list (plugin -> plugins that depend on it) val dependents = plugins.values.foldLeft(Map.empty[PluginId, Set[PluginId]]) { (acc, manifest) => manifest.dependencies.keys.foldLeft(acc) { (a, depId) => val parsed = PluginId.parse(depId) a.updated(parsed, a.getOrElse(parsed, Set.empty) + manifest.id) } } // Calculate in-degrees val inDegree = plugins.keys.map { id => id -> plugins(id).dependencies.size }.toMap // Kahn's algorithm @tailrec def process( queue: List[PluginId], result: List[PluginId], degrees: Map[PluginId, Int] ): Either[CyclicDependencyError, List[PluginId]] = queue match case Nil => if result.size == plugins.size then Right(result.reverse) else Left(CyclicDependencyError(plugins.keySet -- result.toSet)) case head :: tail => val newDegrees = dependents.getOrElse(head, Set.empty) .foldLeft(degrees) { (d, dep) => d.updated(dep, d(dep) - 1) } val newReady = dependents.getOrElse(head, Set.empty) .filter(d => newDegrees(d) == 0) .toList .sortBy(id => -plugins(id).priority) // Higher priority first process(tail ++ newReady, head :: result, newDegrees) val initial = plugins.filter(_._2.dependencies.isEmpty).keys.toList .sortBy(id => -plugins(id).priority) process(initial, Nil, inDegree) ``` ### Pattern 5: Hub-Based Event System **What:** Pub/sub events for loose coupling between plugins **When to use:** Plugin A needs to react to Plugin B's actions without direct dependency **Example:** ```scala // Source: ZIO Hub documentation + WinterCMS Event pattern sealed trait SummerEvent object SummerEvent: // Core lifecycle events case class PluginBooted(id: PluginId) extends SummerEvent case class PluginShutdown(id: PluginId) extends SummerEvent // Domain events (plugins define their own) trait DomainEvent extends SummerEvent // Event service trait EventService: def publish[E <: SummerEvent](event: E): UIO[Boolean] def subscribe: ZIO[Scope, Nothing, Dequeue[SummerEvent]] def subscribeFiltered[E <: SummerEvent: ClassTag]: ZIO[Scope, Nothing, Dequeue[E]] object EventService: val live: ZLayer[Any, Nothing, EventService] = ZLayer.scoped { Hub.bounded[SummerEvent](256).map { hub => new EventService: def publish[E <: SummerEvent](event: E): UIO[Boolean] = hub.publish(event) def subscribe: ZIO[Scope, Nothing, Dequeue[SummerEvent]] = hub.subscribe def subscribeFiltered[E <: SummerEvent: ClassTag]: ZIO[Scope, Nothing, Dequeue[E]] = hub.subscribe.map(_.filterOutput { case e: E => Some(e) case _ => None }) } } // Plugin subscribing to events def boot: ZIO[PluginEnv, PluginError, Unit] = ZIO.serviceWithZIO[EventService] { events => events.subscribeFiltered[UserRegistered].flatMap { queue => queue.take.flatMap { event => // Send welcome email when user registers sendWelcomeEmail(event.user) }.forever.fork } } ``` ### Pattern 6: Type-Safe Extension Traits **What:** Plugins declare extension points as traits, other plugins implement them **When to use:** Plugin A wants to allow Plugin B to extend its functionality **Example:** ```scala // Source: Scala 3 type classes + CONTEXT.md extension mechanics // Blog plugin declares extension point trait BlogExtension: def extendPostFields: List[FieldDef] def extendPostModel(post: Post): Post def validatePost(post: Post): ZIO[Any, ValidationError, Unit] // SEO plugin implements extension object SeoExtension extends BlogExtension: def extendPostFields: List[FieldDef] = List( FieldDef("meta_title", FieldType.Text, tab = "SEO"), FieldDef("meta_description", FieldType.Textarea, tab = "SEO"), FieldDef("canonical_url", FieldType.Text, tab = "SEO") ) def extendPostModel(post: Post): Post = post.copy(extraData = post.extraData ++ Map( "meta_title" -> post.title.take(60), "meta_description" -> post.excerpt.take(160) )) def validatePost(post: Post): ZIO[Any, ValidationError, Unit] = ZIO.when(post.extraData.get("meta_title").exists(_.length > 60)) { ZIO.fail(ValidationError("Meta title must be 60 characters or less")) }.unit // Blog plugin collects all extensions class BlogPlugin extends SummerPlugin: def boot: ZIO[PluginEnv, PluginError, Unit] = ZIO.serviceWithZIO[ExtensionRegistry] { registry => registry.getExtensions[BlogExtension].flatMap { extensions => // Merge all field definitions val allFields = extensions.flatMap(_.extendPostFields) registerFormFields("post", allFields) } } ``` ### Pattern 7: Registration-Based UI Extensions **What:** Plugins register navigation, forms, settings via declarative API **When to use:** Admin UI customization **Example:** ```scala // Source: WinterCMS pattern adapted to Scala def register(ctx: PluginContext): PluginRegistration = PluginRegistration( navigation = List( NavigationItem( id = "blog", label = "lang.blog.menu_label", url = "/admin/golem15/blog/posts", icon = "pencil", permissions = List("golem15.blog.*"), order = 300, sideMenu = List( NavigationItem( id = "posts", label = "lang.blog.posts", url = "/admin/golem15/blog/posts", icon = "copy" ), NavigationItem( id = "categories", label = "lang.blog.categories", url = "/admin/golem15/blog/categories", icon = "list-ul" ) ) ) ), permissions = List( Permission("golem15.blog.access_posts", "Access posts"), Permission("golem15.blog.manage_settings", "Manage settings") ), settings = List( SettingDef( key = "blog", label = "Blog Settings", modelClass = classOf[BlogSettings], icon = "pencil" ) ) ) ``` ### Anti-Patterns to Avoid - **Direct plugin-to-plugin imports:** Use events or extension traits, not direct class imports - **Synchronous boot effects:** Boot must return `ZIO`, never block - **Mutable shared state:** Use `Ref` for concurrent state, never `var` - **Ignoring boot failures:** Failed plugins must disable dependents, not crash app - **String-based event names:** Use sealed trait ADT for events, compile-time safety ## Don't Hand-Roll Problems that look simple but have existing solutions: | Problem | Don't Build | Use Instead | Why | |---------|-------------|-------------|-----| | YAML parsing | Manual string parsing | circe-yaml | Edge cases (quotes, escapes, multiline), type safety | | Version constraints | String comparison | semver-parser-scala | Caret, tilde, range operators are complex | | Topological sort | Naive recursion | Kahn's algorithm or sciss/topology | Cycle detection, efficiency | | File watching | Thread.sleep polling | directory-watcher | OS-native events, efficiency, cross-platform | | Pub/sub events | Callback lists | ZIO Hub | Back-pressure, concurrency, type safety | | Plugin state | Mutable var | ZIO Ref | Atomic updates, concurrent safety | **Key insight:** Plugin systems seem simple but have many edge cases around ordering, failures, and concurrency. ZIO's primitives (Ref, Hub, ZLayer) solve these elegantly. ## Common Pitfalls ### Pitfall 1: Circular Dependencies Not Detected Early **What goes wrong:** Plugin A requires B, B requires A, discovered at boot time causing crash **Why it happens:** Dependencies checked lazily during boot instead of at discovery **How to avoid:** Run topological sort at discovery time, fail fast with clear error message **Warning signs:** "Plugin X is already booting" errors, stack overflows ### Pitfall 2: Partial Boot State **What goes wrong:** Plugin boots halfway, crashes, leaves system in inconsistent state **Why it happens:** Boot effects not atomic, no rollback mechanism **How to avoid:** Use `acquireRelease` pattern - register cleanup in reverse order **Warning signs:** Orphan event listeners, resource leaks, duplicate registrations on retry ### Pitfall 3: Extension Ordering Conflicts **What goes wrong:** Plugin A and B both extend form field "title", last one wins unexpectedly **Why it happens:** Extensions merged without conflict detection **How to avoid:** Detect same-key extensions, error or use explicit priority **Warning signs:** Missing form fields, unexpected field values ### Pitfall 4: Dev Mode Hot-Reload Breaking State **What goes wrong:** Plugin reloaded, but old event listeners still active **Why it happens:** Hot-reload creates new instance without cleanup **How to avoid:** Full shutdown sequence before reload, track all subscriptions in Scope **Warning signs:** Duplicate event handling, ghost listeners, memory growth ### Pitfall 5: Optional Dependency Load Order **What goes wrong:** Plugin loads before its optional dependency, extension not applied **Why it happens:** Optional deps not in topological sort **How to avoid:** Include optional deps in sort but tolerate missing **Warning signs:** Features randomly missing depending on load order ### Pitfall 6: Version Constraint Mismatch **What goes wrong:** Plugin requires `^1.2.0`, installed is `1.1.5`, cryptic runtime errors **Why it happens:** Version constraints not validated at discovery **How to avoid:** Validate all version constraints before any plugin boots **Warning signs:** NoSuchMethodError, ClassNotFoundException at runtime ## Code Examples Verified patterns from official sources: ### Plugin Discovery Service ```scala // Source: ZIO Service Pattern import java.nio.file.{Files, Path, Paths} import scala.jdk.CollectionConverters.* trait PluginDiscovery: def discover(pluginsDir: Path): Task[List[DiscoveredPlugin]] object PluginDiscovery: val live: ZLayer[Any, Nothing, PluginDiscovery] = ZLayer.succeed { new PluginDiscovery: def discover(pluginsDir: Path): Task[List[DiscoveredPlugin]] = ZIO.attemptBlocking { // Find all plugin.yaml files Files.walk(pluginsDir, 3).iterator.asScala .filter(_.getFileName.toString == "plugin.yaml") .toList }.flatMap { manifests => ZIO.foreach(manifests) { path => for content <- ZIO.attemptBlocking(Files.readString(path)) manifest <- ZIO.fromEither(parseManifest(content)) .mapError(e => new Exception(s"Invalid manifest at $path: $e")) yield DiscoveredPlugin(path.getParent, manifest) } } } case class DiscoveredPlugin( directory: Path, manifest: PluginManifest ) ``` ### Version Constraint Validation ```scala // Source: semver-parser-scala documentation import semver._ def validateDependencies( available: Map[PluginId, String], // id -> version required: Map[String, String] // "vendor.name" -> "^1.2.0" ): Either[DependencyError, Unit] = required.toList.traverse { case (idStr, constraint) => val id = PluginId.parse(idStr) available.get(id) match case None => Left(DependencyError.Missing(id)) case Some(versionStr) => val version = SemVer(versionStr) val range = Range(constraint) if range.valid(version) then Right(()) else Left(DependencyError.VersionMismatch(id, constraint, versionStr)) }.void ``` ### Hot-Reload with Directory Watcher ```scala // Source: directory-watcher API import io.methvin.watcher.{DirectoryWatcher, DirectoryChangeEvent} trait HotReloadService: def watchPlugins: ZIO[Scope, Nothing, Unit] object HotReloadService: def live(pluginsDir: Path): ZLayer[PluginManager, Nothing, HotReloadService] = ZLayer.fromFunction { (manager: PluginManager) => new HotReloadService: def watchPlugins: ZIO[Scope, Nothing, Unit] = ZIO.acquireRelease( ZIO.attemptBlocking { DirectoryWatcher.builder() .path(pluginsDir) .listener { event => // Trigger reload on plugin.yaml or .scala changes if event.path.toString.endsWith(".yaml") || event.path.toString.endsWith(".scala") then // Extract plugin ID from path val pluginId = extractPluginId(event.path) Runtime.default.unsafe.run( manager.reloadPlugin(pluginId) ) } .build() }.orDie )(watcher => ZIO.attemptBlocking(watcher.close()).orDie) .flatMap(w => ZIO.attemptBlocking(w.watch()).orDie.fork.unit) } ``` ### Extension Registry ```scala // Source: ZIO Service Pattern + Scala 3 ClassTag import scala.reflect.ClassTag trait ExtensionRegistry: def register[E: ClassTag](pluginId: PluginId, extension: E): UIO[Unit] def getExtensions[E: ClassTag]: UIO[List[E]] def getExtension[E: ClassTag](pluginId: PluginId): UIO[Option[E]] object ExtensionRegistry: val live: ZLayer[Any, Nothing, ExtensionRegistry] = ZLayer.fromZIO { Ref.make(Map.empty[String, Map[PluginId, Any]]).map { registry => new ExtensionRegistry: private def key[E: ClassTag]: String = summon[ClassTag[E]].runtimeClass.getName def register[E: ClassTag](pluginId: PluginId, extension: E): UIO[Unit] = registry.update { map => val k = key[E] val existing = map.getOrElse(k, Map.empty) map.updated(k, existing + (pluginId -> extension)) } def getExtensions[E: ClassTag]: UIO[List[E]] = registry.get.map { map => map.getOrElse(key[E], Map.empty) .values.toList.asInstanceOf[List[E]] } def getExtension[E: ClassTag](pluginId: PluginId): UIO[Option[E]] = registry.get.map { map => map.getOrElse(key[E], Map.empty) .get(pluginId).asInstanceOf[Option[E]] } } } ``` ## State of the Art | Old Approach | Current Approach | When Changed | Impact | |--------------|------------------|--------------|--------| | Runtime reflection for plugins | Compile-time traits + circe derivation | Scala 3 | Type safety, better errors | | Callback-based events | ZIO Hub pub/sub | ZIO 2.0 | Back-pressure, concurrency safety | | Mutable plugin state | ZIO Ref | ZIO 2.0 | Atomic updates, no race conditions | | Manual dependency ordering | Topological sort at discovery | Always best practice | Fail-fast on cycles | | Polling for file changes | directory-watcher native events | 2020+ | Efficiency, responsiveness | **Deprecated/outdated:** - Java ServiceLoader: Runtime reflection, no Scala 3 native support, hard to configure - Akka actors for events: Overkill, ZIO Hub is simpler and sufficient - YAML via SnakeYAML direct: Requires JavaBeans, less idiomatic for Scala ## Open Questions Things that couldn't be fully resolved: 1. **Shutdown Timeout Policy** - What we know: Plugins need graceful shutdown, but can hang - What's unclear: What timeout is appropriate, should it be configurable? - Recommendation: Default 30 seconds per plugin, configurable in manifest, force-kill after timeout 2. **Event Listener Priority** - What we know: Multiple plugins may listen to same event - What's unclear: Should listeners have ordering? What if one listener should cancel event? - Recommendation: Ordered by plugin dependency (dependents first), no cancellation in v1 3. **Plugin Isolation Level** - What we know: Plugins share JVM, no classloader isolation - What's unclear: Can a plugin crash the whole app? - Recommendation: Use ZIO error handling, defects propagate but are logged, continue running 4. **YAML vs Code for Form Extensions** - What we know: CONTEXT.md says hybrid (simple in YAML, complex in code) - What's unclear: Exact boundary, how YAML extensions merge with code - Recommendation: Define clear merge rules - YAML first, code can override/extend ## Sources ### Primary (HIGH confidence) - [ZIO Hub Documentation](https://zio.dev/reference/concurrency/hub/) - Pub/sub pattern - [ZIO Ref Documentation](https://zio.dev/reference/concurrency/ref/) - State management - [ZIO Scope Documentation](https://zio.dev/reference/resource/scope/) - Resource lifecycle - [ZIO Architectural Patterns](https://zio.dev/reference/architecture/architectural-patterns/) - Service layer design - [circe-yaml GitHub](https://github.com/circe/circe-yaml) - YAML parsing, version 0.16.1 - [directory-watcher GitHub](https://github.com/gmethvin/directory-watcher) - File watching, version 0.19.1 ### Secondary (MEDIUM confidence) - [sciss/topology Scaladex](https://index.scala-lang.org/sciss/topology) - Topological sort library - [semver-parser-scala GitHub](https://github.com/sh4869/semver-parser-scala) - Version constraint parsing - [SoftwareMill ZIO 2 Structure](https://softwaremill.com/structuring-zio-2-applications/) - Service patterns - [Scala 3 Extension Methods](https://docs.scala-lang.org/scala3/book/ca-extension-methods.html) - Type-safe extensions ### Tertiary (LOW confidence) - [ZIO-NIO Archive](https://github.com/zio-archive/zio-nio) - Archived in Jan 2025, patterns still valid - [Topological Sort Guide](https://medium.com/@amit.anjani89/topological-sorting-explained-a-step-by-step-guide-for-dependency-resolution-1a6af382b065) - Algorithm explanation ## Metadata **Confidence breakdown:** - Standard stack: HIGH - All libraries verified via official docs and releases - Architecture patterns: HIGH - Based on ZIO official documentation and WinterCMS reference - Pitfalls: MEDIUM - Mix of ZIO docs and WinterCMS experience - Extension API: MEDIUM - Novel design combining multiple patterns **Research date:** 2026-02-05 **Valid until:** 2026-03-05 (30 days - stable patterns, libraries may update)