golem15/summercms-initial-research
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1ad199363232f8f82de9eb35d27d7e42e2b11ef6
summercms-initial-research/.planning/research/PITFALLS.md
Jakub Zych a185132635 Initial research ahahahah
2026-02-04 02:18:31 +01:00

22 KiB
Raw Blame History

Pitfalls Research: SummerCMS

Project: SummerCMS - Scala/ZIO rewrite of WinterCMS Researched: 2026-02-04 Confidence: MEDIUM-HIGH (verified via multiple sources)

Executive Summary

This research identifies critical pitfalls across four dimensions: Scala/ZIO technical challenges, CMF rewrite patterns, scope/ambition management, and team/process dynamics. The most dangerous pitfalls are:

  1. The Big Rewrite Anti-Pattern - Trying to achieve full parity before shipping anything
  2. ZIO Learning Curve for Non-FP Developers - 40.5% of teams report slowed onboarding
  3. AI Agent Quality Debt - Research shows AI assistance creates persistent technical debt
  4. Plugin System Design Lock-in - Getting the API wrong forces breaking changes

Scala/ZIO Specific Pitfalls

1. Incorrectly Wrapping Async Code with ZIO.effect

What goes wrong: When wrapping legacy or third-party APIs, developers use ZIO.effect (now ZIO.attempt) for code that returns Future. The ZIO effect completes without waiting for the Future, causing race conditions and silent failures.

Warning signs:

  • Race conditions in async operations
  • "Impossible" null pointer exceptions
  • Tests passing locally but failing in CI
  • Side effects executing out of order

Prevention strategy:

  • Use ZIO.fromFuture for Future-returning code
  • Create team training on ZIO effect wrapping patterns
  • Add lint rules or code review checklist item
  • Establish wrapper utility library early

Phase relevance: Phase 1 (Core Framework) - must be addressed in initial architecture

2. FiberFailure Exception Wrapping

What goes wrong: When using Runtime.unsafeRun, ZIO throws FiberFailure wrapping the original exception. Code expecting to catch specific exceptions (e.g., SQLException) fails because only FiberFailure propagates.

Warning signs:

  • Exception handlers not triggering
  • Generic error messages losing original cause
  • Interop with Java libraries failing unexpectedly

Prevention strategy:

  • Use Cause.squash when extracting errors for interop
  • Document exception unwrapping patterns
  • Create standardized error handling utilities for Java interop

Phase relevance: Phase 1-2, especially when integrating database and external services

3. Unintentional Side Effects Outside ZIO

What goes wrong: Developers add println statements or other side effects in val definitions without wrapping in ZIO. These execute eagerly during effect construction, causing out-of-order execution.

Warning signs:

  • Log statements appearing in wrong order
  • "Debug" side effects happening at import time
  • Confusion about when code actually runs

Prevention strategy:

  • Use ZIO.logDebug instead of println
  • Configure linter to warn on println usage
  • Code review focus on val definitions

Phase relevance: All phases - cultural issue to address from day 1

4. ZIO HTTP Client Performance Overhead

What goes wrong: ZIO HTTP Client has ~2 second initialization overhead and ~300ms per-request overhead without connection pooling. Default configuration is not production-ready.

Warning signs:

  • API response times 300ms+ slower than expected
  • Memory growing with request count
  • Service restarts due to memory exhaustion

Prevention strategy:

  • Always use connection pooling (Client.default with pool)
  • Configure client initialization at app startup, not per-request
  • Monitor memory and finalizer counts
  • Set up production load testing early

Phase relevance: Phase 2-3 when building HTTP layer

Sources: ZIO HTTP Client Performance Issue #2117, Memory Leaks Issue #2956

5. ZIO Quill Compile-Time Explosion

What goes wrong: Projects with many Quill queries see compile times grow exponentially. Single queries can take 5+ minutes and require 5GB+ heap. This kills developer productivity.

Warning signs:

  • Compile times increasing each sprint
  • CI builds timing out
  • Developers avoiding database code changes
  • "Slow inlining phase" in compiler output

Prevention strategy:

  • Split database layer into separate sbt subproject
  • Limit queries per file (suggested: 10-15 max)
  • Use scalacOptions += "-Yretain-trees" to avoid intermittent failures
  • Configure CI with sufficient memory (8GB+ for Quill projects)
  • Consider Doobie as alternative for complex query scenarios

Phase relevance: Phase 2 (Database Layer) - architecture decision point

Sources: Quill Compilation Time #2737, ZIO Quill Compiler Performance

6. Infinite Compile Time with ZLayer.fromFunction

What goes wrong: When ZLayer.fromFunction receives more type parameters than needed, the compiler enters infinite loop until heap exhaustion.

Warning signs:

  • Compilation never finishing
  • OutOfMemoryError during build
  • CI job killed after timeout

Prevention strategy:

  • Use ZLayer.derive for automatic layer construction
  • Provide explicit type annotations on ZLayer definitions
  • Code review for ZLayer boilerplate

Phase relevance: Phase 1 (Core Framework) - affects DI architecture

Sources: ZIO Issue #8701

7. Overlapping Routes with Tapir/ZIO-HTTP

What goes wrong: ZIO HTTP's routing implementation cannot correctly handle overlapping path shapes. Routes like /users/{id} and /users/admin conflict, causing 404 or 405 errors.

Warning signs:

  • Intermittent 404s on valid endpoints
  • Header-based versioning not working
  • First-listed route capturing all traffic

Prevention strategy:

  • Design non-overlapping URL structures
  • Avoid header-based routing for endpoint selection
  • Use path prefixes for API versioning (/v1/, /v2/)
  • Test all route combinations explicitly

Phase relevance: Phase 2-3 (API Layer) - design routing conventions early

Sources: Tapir ZIO HTTP Docs

8. Misusing ZIO Environment for Service Passing

What goes wrong: Teams use ZIO's R environment to pass services around like a DI container, creating complex type signatures and making code hard to follow.

Warning signs:

  • ZIO effect types with 5+ services in R
  • Difficulty understanding what services are needed where
  • Excessive provideSome and type manipulation

Prevention strategy:

  • Use constructor injection for services
  • Reserve R environment for contextual needs (scopes, transactions)
  • Follow pattern: "constructors require, layers make"
  • Use ZLayer.derive for automatic wiring

Phase relevance: Phase 1 (Core Framework) - establish patterns before codebase grows

Sources: Structuring ZIO 2 Applications

CMF Rewrite Pitfalls

9. The Iceberg Underestimation

What goes wrong: Product managers and developers see the visible UI/UX of WinterCMS and assume that's the scope. But like an iceberg, only 10% is visible. The migration system, YAML parsing edge cases, plugin lifecycle hooks, and error recovery paths are the hidden 90%.

Warning signs:

  • Estimates based on feature count, not implementation depth
  • "It's just a CRUD form generator" mentality
  • Discovery of essential features mid-implementation
  • Timeline constantly extending

Prevention strategy:

  • Inventory ALL WinterCMS features before scoping (use reference implementation)
  • Weight estimates by implementation complexity, not user visibility
  • Build explicit unknown-unknowns buffer (30-50% for Phase 1)
  • Accept that Phase 1 will reveal scope for later phases

Phase relevance: All phases - especially Phase 1 planning

Sources: Why Rewrites Fail

10. Moving Target Syndrome

What goes wrong: The original WinterCMS continues evolving while SummerCMS is built. Teams chase feature parity with a moving target, implementing features twice or falling perpetually behind.

Warning signs:

  • WinterCMS releases during development causing re-work
  • "We need to add X because they just added it" mid-sprint
  • Never reaching parity point
  • Demoralization from endless chase

Prevention strategy:

  • Define parity target against specific WinterCMS version (freeze reference)
  • Accept that SummerCMS 1.0 will differ from WinterCMS latest
  • Focus on core value proposition, not feature-for-feature matching
  • Plan "catch-up" sprints explicitly, not continuously

Phase relevance: Project-wide - establish as principle in Phase 0

11. Plugin API Lock-in

What goes wrong: The plugin system API is designed early but proves inadequate. Changing it later breaks all plugins, requiring mass rewrites or painful deprecation cycles.

Warning signs:

  • Plugin developers working around API limitations
  • Plugins accessing internal APIs
  • Feature requests that require plugin API changes
  • "We can't do X because of the plugin contract"

Prevention strategy:

  • Study multiple plugin system designs (WordPress, WinterCMS, Discourse, VSCode)
  • Build 3-5 diverse plugins before finalizing API
  • Use semantic versioning from day 1
  • Design for extension points (hooks, filters, traits)
  • Plan for plugin API v2 from the start (versioned endpoints)

Phase relevance: Phase 3-4 (Plugin System) - most critical architecture decision

12. YAML Parser Edge Cases

What goes wrong: WinterCMS's YAML-driven forms have subtle behaviors and edge cases. A naive reimplementation misses validation cascades, conditional visibility, custom widget loading, and translation lookups.

Warning signs:

  • Forms "mostly work" but have weird bugs
  • Copy-pasting YAML from WinterCMS requires modifications
  • Custom widgets not loading correctly
  • Translations missing in form contexts

Prevention strategy:

  • Create comprehensive test suite from WinterCMS YAML examples
  • Document all YAML features before implementation
  • Build incrementally with full test coverage at each step
  • Consider typed config format (HOCON, Dhall) as alternative

Phase relevance: Phase 2-3 (Backend/Admin System)

13. Authentication/Authorization Reimplementation

What goes wrong: Authentication seems simple until you hit: password reset flows, account lockout, session management, remember-me tokens, OAuth providers, JWT refresh, GDPR deletion requirements, and audit logging.

Warning signs:

  • "Just build login first, we'll add security later"
  • Security review finding critical issues
  • Edge cases discovered in production
  • Compliance requirements appearing late

Prevention strategy:

  • Use battle-tested library (ZIO-Auth, or wrap proven Java library)
  • Define security requirements before implementation
  • Include compliance checklist (GDPR, CCPA) in design
  • Penetration testing in Phase 2, not after launch

Phase relevance: Phase 2 (Auth/User System) - do not underestimate

14. Migration Tool Neglect

What goes wrong: Teams focus on new CMS but neglect the migration path from WinterCMS. Users can't adopt because they can't migrate their existing sites.

Warning signs:

  • "Migration is a post-launch concern"
  • Data model designed without migration in mind
  • Plugin compatibility assumptions
  • No testing with real WinterCMS sites

Prevention strategy:

  • Include migration as Phase 2/3 requirement, not post-launch
  • Test with 3-5 real WinterCMS sites during development
  • Document breaking changes and migration paths
  • Build automated migration tooling

Phase relevance: Phase 4-5 - but requires design consideration in Phase 1

Scope/Ambition Pitfalls

15. The "Full Parity" Delusion

What goes wrong: Targeting "full WinterCMS parity" sounds achievable but is actually unbounded scope. WinterCMS has 15+ years of features, edge cases, and workarounds. Full parity could take 5+ years.

Warning signs:

  • Feature list keeps growing
  • No clear definition of "parity"
  • Blocking launch on nice-to-have features
  • Team fatigue from endless scope

Prevention strategy:

  • Define "Essential Parity" vs "Nice-to-Have Parity"
  • Create tiered launch criteria:
    • MVP: Build a new site
    • v1.0: Migrate simple WinterCMS site
    • v1.5: Migrate complex WinterCMS site
  • Accept 80/20 rule: 80% of value from 20% of features
  • Document what's explicitly NOT in v1.0

Phase relevance: Phase 0 (Planning) - critical scope decision

16. Second System Effect

What goes wrong: Having learned from WinterCMS's limitations, the team over-engineers SummerCMS with every feature they wished the original had. The result is bloated, slow, and late.

Warning signs:

  • "While we're at it, let's also..."
  • Architecture astronaut discussions
  • Features justified by "what if" scenarios
  • Premature optimization before working system

Prevention strategy:

  • Limit architectural innovations per phase (suggest: 1-2 max)
  • Require user story for every feature
  • "Build the simplest thing that could possibly work" as team mantra
  • Technical debt is acceptable in v1; perfection is not required

Phase relevance: All phases - ongoing discipline

Sources: The Mythical Man-Month (Fred Brooks)

17. Technology FOMO

What goes wrong: During multi-year development, new technologies emerge. Teams want to switch to the latest ZIO version, new HTTP libraries, or shiny new patterns. Constant upgrades destabilize the codebase.

Warning signs:

  • "ZIO X.Y just released, we should upgrade"
  • Multiple active branches for different tech stacks
  • Library upgrade PRs exceeding feature PRs
  • Breaking changes from dependency updates

Prevention strategy:

  • Freeze major dependencies for 6-month windows
  • Upgrade in dedicated sprints, not continuously
  • Maintain changelog awareness but resist immediate adoption
  • Prefer stable over bleeding-edge for core dependencies

Phase relevance: All phases - establish upgrade policy in Phase 1

18. Parallel Development Without Integration Points

What goes wrong: Teams work on plugin system, admin UI, and API layer in parallel but without regular integration. When they merge, nothing fits together.

Warning signs:

  • Long-running feature branches
  • Merge conflicts taking days to resolve
  • "Demo day" disasters
  • Differing assumptions about shared interfaces

Prevention strategy:

  • Weekly integration builds mandatory
  • Shared interface contracts (API specs, type definitions) before implementation
  • Vertical slices: build thin complete features, not horizontal layers
  • CI must run integration tests on all PRs

Phase relevance: All phases - process discipline

Team/Process Pitfalls

19. AI Agent Quality Debt Accumulation

What goes wrong: Research shows AI-assisted coding creates "substantial but transient velocity gains alongside persistent increases in technical debt." The initial speed comes at the cost of code quality that compounds over time.

Warning signs:

  • PRs with unusual patterns or verbose code
  • Test coverage declining despite more code
  • Static analysis warnings increasing
  • Experienced developers spending time debugging AI code
  • "The agent wrote it, I don't fully understand it"

Prevention strategy:

  • Mandatory human review for all AI-generated code
  • AI agents work on tests FIRST, then implementation
  • Static analysis gates in CI (no warning increase allowed)
  • Limit AI to well-defined, isolated tasks
  • "Agent code" labeled for extra scrutiny

Phase relevance: All phases - process constraint

Sources: Speed at Cost of Quality Study, METR AI Developer Study

20. AI Confirmation Bias

What goes wrong: AI agents affirm user premises even when wrong. Developers ask "Is this design correct?" and get "Yes!" even for flawed designs.

Warning signs:

  • Lack of pushback on technical decisions
  • Bugs in areas "validated" by AI
  • False confidence in untested approaches

Prevention strategy:

  • Use AI for implementation, not architecture validation
  • Human architect reviews all design decisions
  • Ask AI adversarial questions: "What could go wrong with this?"
  • Require AI to cite sources for technical claims

Phase relevance: All phases - especially architecture phases

Sources: VentureBeat on AI Agent Limitations

21. Scala Learning Curve for Web Developers

What goes wrong: Target users are "web developers who may not know Scala deeply," but the codebase uses advanced ZIO patterns, tagless final, and complex type signatures. Contributors can't understand or extend the system.

Warning signs:

  • External PRs take weeks to review due to errors
  • Same questions asked repeatedly in issues
  • Plugin developers working in different style than core
  • 40.5% onboarding slowdown (industry research)

Prevention strategy:

  • Create "SummerCMS Style Guide" limiting advanced patterns
  • Use simple, explicit types over clever abstractions
  • Extensive inline documentation for non-obvious code
  • "Stepping project" for new contributors (simple bugs first)
  • Consider Scala 3 direct-style syntax over ZIO everywhere

Phase relevance: Phase 1 - establish code style early

Sources: Xebia: Onboarding Juniors into Scala, State of Scala Survey

22. Expert Blindness on Complexity

What goes wrong: The principal Scala developer builds elegant, idiomatic code that only they can maintain. When they're unavailable, development stops.

Warning signs:

  • Only one person can debug certain modules
  • Code reviews always require expert input
  • "Ask [expert]" as common answer to questions
  • Bus factor of 1 for critical systems

Prevention strategy:

  • Pair programming on complex modules
  • All code must be explainable to non-expert in 5 minutes
  • Rotate ownership of modules
  • Document "why" not just "what" in code comments

Phase relevance: All phases - team management

23. Cake Pattern and Implicit Abuse

What goes wrong: Teams use Scala's Cake pattern for dependency injection, creating unmaintainable "hairball" of traits. Or they overuse implicits until nobody can trace where values come from.

Warning signs:

  • Traits with 10+ dependencies
  • implicit parameters that could be explicit
  • IDE unable to find usages
  • New developers completely lost

Prevention strategy:

  • Use ZLayer for DI, not Cake pattern
  • Prefer explicit constructor parameters over implicits
  • Limit implicit scope to truly contextual needs
  • IDE-friendly code as explicit goal

Phase relevance: Phase 1 - architecture decision

Sources: Scala Best Practices - Architecture

Critical "Do Not" List

These are absolute no-gos that lead to project failure:

DO NOT: Build Without a Working Prototype First

Build a minimal end-to-end prototype (CLI -> HTTP -> DB -> Response) before any "real" development. This validates the stack integration and uncovers tooling issues.

DO NOT: Design Plugin API in Isolation

Build 3+ real plugins before finalizing the plugin API. Paper designs miss critical use cases.

DO NOT: Skip Type Safety for "Simplicity"

Scala's value is in type safety. Cutting corners with Any or stringly-typed code creates bugs that surface in production.

DO NOT: Ignore Compile Time Budgets

Set compile time budgets per module (e.g., 30 seconds). When exceeded, split the module. Slow compilation kills productivity.

DO NOT: Let AI Agents Work Without Tests

Require tests BEFORE implementation. AI-generated code without tests is tech debt disguised as velocity.

DO NOT: Promise WinterCMS Compatibility Without Testing It

Every compatibility claim must have a passing test against real WinterCMS artifacts.

DO NOT: Defer Security to "After Launch"

Authentication, authorization, and input validation are Phase 1 concerns, not Phase N polish.

DO NOT: Build Horizontal Layers in Sequence

Don't build all database layer, then all API layer, then all UI. Build vertical slices that prove integration works.

Pitfall-to-Phase Mapping

Pitfall Primary Phase Mitigation Phase
ZIO.effect misuse 1 Code review from day 1
Fiber exception wrapping 1-2 Utility library in Phase 1
ZIO HTTP performance 2-3 Load testing in Phase 2
Quill compile times 2 Architecture decision
Overlapping routes 2-3 URL convention in Phase 2
Plugin API lock-in 3-4 Build plugins before finalizing
YAML edge cases 2-3 Test suite from WinterCMS
Auth complexity 2 Use proven library
AI quality debt All CI gates + review process
Learning curve All Style guide + onboarding
Moving target All Freeze reference version
Full parity delusion 0 Scope discipline

Sources

ZIO/Scala Technical

Rewrite/CMF

AI Development

Scala DX/Onboarding

Reference in New Issue View Git Blame Copy Permalink