Hacker News new | ask | show | jobs
by modeless 2078 days ago
I found python multiprocessing to work well to parallelize deep learning data loading and preprocessing, because all I needed to communicate was a couple of tensors which are easy to allocate in shared memory. I didn't need complex data structures or synchronization.