|
|
|
|
|
by barrkel
4700 days ago
|
|
Need to download a bunch of files from a remote host, process them, and write them to disk? cat urls.txt | while read -r url; do
base="$(basename "$url")"
wget "$url" -O - -o "$base.log" | process > "$base" &
done
wait
For ad-hoc scripting, I tend to prefer bash, with anything remotely corresponding to heavy lifting assigned to the language with the best library / performance / whatever critical factor for it. Fork-join parallelism in bash is very easy. |
|