Hacker News new | ask | show | jobs
by mynewtb 2977 days ago
Sure thing...

    from multiprocessing import Pool
    with Pool(8) as p:
        p.map(function, sequence))
2 comments

Neat.

I see that you explicitly specify 8 as the number of processes here. In Java, parallelStream() will pick a sane default for you if you haven't previously specified (based on the number of available processors, I believe). Is something like that possible in Python?

The ThreadPool class picks a sane default (number of cores), but I believe it uses Python threads instead of processes.
You could just use threads here too - CPython releases the GIL during IO.

  from concurrent.futures import ThreadPoolExecutor
  with ThreadPoolExecutor as executor:
      executor.map(function, sequence)