[miiike]

I wanted to ask you, what about running multiple operations in parallel? I've long been contemplating async/futures vs lightweight threads and yet to have come to a conclusion.

With futures I can say, start operation 1 and operation 2 in parallel, then chain a callback to execute using both pieces of data, saving me some latency of doing the operations serially. How do you do this using quasar? Now that's a trivial example... what about arbitrary dependency trees? This falls out naturally with futures but I don't know of a nice way to do this with lightweight threads. e.g. 1 operation branches off into 5 other parallel operations which then chain some of their own additional callbacks for processing before finally bringing all the results back together, perhaps to form a json response. Does this example make sense?

[pron]

With lightweight threads you can spawn as many fibers as you like. Creating and starting a new fiber is basically free. You can start fibers and join them, in any dependency tree structure.

Of course, you can keep using futures (what I call semi-blocking API), only futures that block the fiber rather than the thread, when you join them.

[miiike]

Okay so to run operations in parallel you have to go back to futures, and for more complex dependencies you would need callbacks/transforms. So this means with fibres you could use the simpler synchronous model for serial operations, but future model for parallelism, i.e. a hybrid model. My thoughts are that it might be simpler to adopt a single model rather than two. On the flip side you could argue that with fibres you don't need to use the more complicated parallelism model all the time and only when needed.

[pron]

If you want to run operations in parallel and then "join" them, you need some joining mechanism. A fiber itself can be joined and return a value, so in Quasar, Fiber implements Future. But that's semantics: with fibers, if you want to run operations in parallel, you spawn more fibers; if you then want to join those operations -- you join the fibers.

Of course you can have long-running fibers that interact with one another through channels. See: http://blog.paralleluniverse.co/2014/02/20/reactive/

[miiike]

Thanks, this is the kind of answer I was looking for. I'll have a read once I'm on a bigger screen.

[runT1ME]

Seriously, just use scala Futures. Here's one of many super easy ways to do parallel computations in scala: val list = getItemsToProcess() //List[SomeObj] list.map(so => Future { processorHeavyMethod(obj) }) //happening in parallel now val finished = Future.sequence(list) //Future[List] finished.await