[dxbydt]

I've found that if the list is very large, using list.par.map(...) hits the memory limit & then simply hangs.

I process upwards of a million facilities ( a facility is the exposure on a commercial loan a bank makes to a client ), trying to forecast their expected loss over 12 future quarters.

I rewrote my code like so :

val facs:List[Facility] = ...populated via jdbc query ///

val N = 1000

facs.grouped(N).foreach( group => {

   group.par.map( fac => expectedloss(fac)

})

Now the outer loop is sequential but the inner loop is parallel. It chugs along, and processes the entire million plus facilities! I get some control via the N. Currently my N is 1000 and all 8 cores running at 100% capacity, but I can slow down the workload with smaller N.