[visarga]

I write complex shell commands every day, but when it gets longer than 2-3 rows I switch to a text editor and write it in Perl instead. I see no need to use bash up to that complexity, doesn't look good in terminal.

Poorman version of multiple pipes is to write intermediate results into files, then "cat" the files as many times as needed for the following processes. I use short file names "o1", "o2" standing for output-1, output-2 and see them as temp variables.

[vinceguidry]

This is what it comes down to to me too. Using the shell to do programming seems to me like putting your job on hard mode.

When I had to do a lot of data processing at my last job, I started building up tools in Ruby. If I had time, I'd hack the workflow so that the next time I needed it, I could just run the tool from the command line.

Eventually I had a pluggable architecture that I could use to pull data from any number of sources and mix it with any other data. Do that with a shell? Why?

[DSpinellis]

The advantage of using the shell are the hundreds of powerful command-line tools you can use. Increasingly, there are Perl/Python/Ruby packages that offer similar functionality, but these require some ceremony to use and therefore prohibit rapid prototyping and experimentation.