[kjkjadksj]

Snakemake is easy to deal with that scenario. I had a profile for each of our slightly different hpc clusters. You could throttle the array by total resources so I could say request no more than 750gb memory allocated across the array to be polite to the rest of the hpc users, and it would fit however many jobs it could within that constraint and step of the pipeline. I could have a job instead be ran on the internet connected head node vs airgapped compute node if something needed downloading. Worked great and the python syntax is pretty useful along with conda env management baked in.