[nop_slide]

That makes sense! I will admit it was a bit of fun clobbering together the bash version at first. It reminded me of starting out programming where everything felt a bit esoteric lol.

> if you had avoided the parallel processing and done it in a more linear way, you've had a better time.

I had to process ~500,000 files, and the aws api call was on average ~1 second, so it would have been a significantly longer time to process linearly. For example the bash version I whipped up processed ~30k files in 2.5 hours, while the python version did 30k files in ~20 minutes.

But yes I agree, if I didn't have to do such a large volume at once the bash version would have been just fine.