[BogusIKnow]

As is sometimes the case, people start with the wrong abstraction (thread pool) and then optimize it.

Java/Scala streams look like they would solve most of the problems in the post on their own.

e.g. Java streams

http://radar.oreilly.com/2015/02/java-8-streams-api-and-para...

    private void runParallel() {
      trades
      .stream()
      .parallel().forEach(t->doSomething(t) );
    }

Java/Scala has a hierarchy of concurrency abstractions, from high level to low level:

Streams, Parallel collections, Futures, Actors, Fork/Join, ExecutionContext/ExecutorService, Threadpools, Threads, synchronized, Barriers, Atomic values and many more.

[BogusIKnow]

Something not so obvious for beginners: Java/Scala does not use all the available memory. If you get out of memory problems with the 10gb data set and you have much more RAM available, you need to tell Java/Scala to increase the Heap.