Benchmarks: Dsl.scala vs Monix vs Cats Effect vs Scalaz Concurrent vs Scala Async vs Scala Continuation

We created some benchmarks to evaluate the computational performance of code generated by our compiler plug-in, especially, we are interesting how our !-notation and other direct style DSL affect the performance in an effect system that support both asynchronous and synchronous effects.

In spite of keywords of adapters to monads or other effect systems (see domain.cats and domain.scalaz), the preferred effect system for Dsl.scala is Task, the type alias of vanilla continuation-passing style function, defined as:

type !![Domain, Value] = (Value => Domain) => Domain
type TaskDomain = TailRec[Unit] !! Throwable
type Task[Value] = TaskDomain !! Value

Our benchmarks measured the performance of Task in Dsl.scala, along with other combination of effect system with direct style DSL, listed in the following table:

The combination of effect system and direct style DSL being benchmarked

Effect System	direct style DSL
Dsl.scala Task, an alias of vanilla continuation-passing style functions	!-notation provided by Dsl.scala
Scala Future	Scala Async
Scala Continuation library	Scala Continuation compiler plug-in
Monix tasks	for comprehension
Cats effects	v comprehension
Scalaz Concurrent	for comprehension

The performance of recursive tasks in each effect systems

The purpose of the first benchmark is to determine the performance of recursive functions in various effect system, especially when a direct style DSL is used.

The performance baseline

In order to measure the performance impact due to direct style DSLs, we have to measure the performance baseline of different effect systems at first. We created some benchmarks for the most efficient implementation of a sum function in each effect system. These benchmarks perform the following computation:

• Creating a List[X[Int]] of 1000 tasks, where X is the data type of task in the effect system.
• Performing recursive right-associated "binds" on each element to add the Int to an accumulator, and finally produce a X[Int] as a task of the sum result.
• Running the task and blocking awaiting the result.

TODO: incomplete