[barrkel]

OTOH, if you can structure your problem as a composition of pipelines, it can be quite a bit faster in bash than in a "proper" language. You get to choose optimized tools, and they run concurrently.

Writing efficient bash code forces you to think about your problem differently. It's a very similar process to thinking functionally; e.g. you don't want to deal with lines of a file one at a time in a loop, you want to do filters and maps in languages like grep, sed and awk to deal with data in a streaming fashion with a minimum of forked processes.

[fiatjaf]

Is there a "proper scripting language" somewhere that supports the same first-class access to Unix programs and the same piping | syntax that shells in general do?

I would love something like that.

[kronos29296]

Perl was created to do this stuff. Perl is like duct tape to hold together everything. Most people call it ugly because of that but Perl is the best language for quick dirty one liners that work. Only problem is that if you look at it later you will have problems understanding it.

[TheGrassyKnoll]

Agree. Perl is terrific for quick text processing. You can make surprisingly powerful one-liners.

[soveran]

In most platforms you can install the rc shell from plan9. It's what I use exclusively for my shell scripts. It is only when I want to share some script with other people that I consider using /bin/sh, and even then I've gone for rc nevertheless. Here you can read about it: http://doc.cat-v.org/plan_9/4th_edition/papers/rc

[chriswarbo]

Thanks for the suggestion. I've also been after a saner shell, and have been disappointed in one way or another with the approach of either using another language idiomatically (Python, Scheme, Haskell, Scala, etc.), since running commands, piping, etc. are all rather awkward; or using such languages with embedded shell-like libraries, which seem to have awkward edge-cases.

As a "real" shell, it looks like rc maintains the command, file and piping niceties of bash, whilst avoiding the edge-cases of shell-like embeddings.

[jerf]

In Perl, IPC::Run [1], while a bit on the quirky side sometimes, can do that sort of thing.

    use IPC::Run qw(run);
    IPC::Run::run(["echo", "abc"], "|", 
                  ["tr", "a-z", "A-Z"], "|", 
                  ["cat"],
                  \$out);
    print "out: $out";

will yield

    out: ABC

Useless use of cat here just to make a point, of course.

It's easy to use Perl strings as input or output, or files, and there's also ways to interact with streams as they go along.

Generally speaking the slightly more verbose interaction with shell is made up for by the savings when you can use Perl directly to do something rather than spawn a shell process for something simple. (Shell composition can be powerful, yes, but on the other hand spawning a full process and setting up a pipe for things like "tr" or "wc" is often just silly.)

I also personally believe there's a win in the syntax; you may say "What? 'tr a-z A-Z' is way more convenient than '["tr", "a-z", "A-Z"]' but I say one of the biggest and most pervasive errors in shell is to incorrectly set up the arguments by having something interpolated in incorrectly. Having an unambiguous syntax for separating arguments has often made it much safer for me to write the code that has to use the shell. Used correctly it can even be made to work in the face of user-supplied input, something that should generally not be combined with any form of implicit argument separation. (Although bear in mind that "used correctly" encompasses more than just "separating the arguments correctly.")

I am also NOT claiming exclusivity; this is just the thing I know off the top of my head. I'm sure the other scripting languages have nice libraries, too. Though watch out for them being too nice. There's a bit of an impedance mismatch between shell and scripting language. Anything that smooths it over too well is probably either giving up critical features or introducing subtle bugs, which can become security bugs in the face of arbitrary user input.

[1]: http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm

[reacweb]

Perl of course, even if the adjective "proper" seems weird for Perl ;-)

As a rule of thumb, if a script starts to need arrays or to handle spaces in filename, I migrate it to Perl.

Error handling is also easier and more natural in Perl.

[fiatjaf]

Just found out that xon.sh http://xon.sh/tutorial . html does this.

[yellowapple]

Tcl lacks the piping syntax by default, but there are various ways to implement it: http://wiki.tcl.tk/17419

[oblio]

Then you have the age old problem: what if one of the tasks in your pipeline(s) fails?

[yorwba]

  set -o pipefail -o errexit

Add more options to taste.

[Benjamin_Dobell]

https://stackoverflow.com/questions/29532904/bash-subshell-e...

Not pipefail as such, but Bash's error handling semantics (or lack there of) is pretty lame.