[Scubabear68]

I don’t think you’ve seen the kind of scripts the person you are responding to is talking about.

I have, and mentioned one lower down in the comments. Unix philosophy was great but does not scale well in terms of maintainability or efficiency. Invoking processes over and over again loops is godawful slow. And the horror of complicated shell scripts is legendary.

[sigg3]

As a self-taught coder, I've experienced many times how highly skilled software engineers groan and sweat when they encounter shell scripts. I don't understand why, but it seems like people with a CS background are never really taught shell scripts and have come to irrationally fear them. It's sort of taboo.

This results in weird behavior, such as writing a groovy (Java?) script for Jenkins to execute bazel in order to build a go binary that runs the very same commands in an exec.Command() construct. Or people who download and import pandas to grab the third field in a csv file.

During the course of learning, I've naturally written code in bash that should have been written in another language. I replaced if statements with case because they turned out to be more performant. It's a great learning experience and why I got into python and go.

IMO we should use the right tool for the job. Sometimes that tool is a combination of unix utilities that you can put in a shell script for easier maintenance. It's just procedural execution of (usually very efficient) binaries, akin to a jenkins script or gitlab pipeline. Just mind the exceptions and use exit codes.

[Scene_Cast2]

I'm the type of person that would grab pandas to parse a CSV. Here's my reasoning

* often times, it's not just the third column I want. Sometimes it becomes "third column unless the first column is 'b' then instead grab the fourth column". Having a good data representation makes sure that I'm not mixing logic code with representation parsing code

* I don't have to care about CSV parsing edge cases. Escaped comma? Quotes? I don't care, the library will either handle it or throw an explicit error. With custom parsing code, instead of an error, I'll get some mangled result in the middle of the file that I won't even catch / notice until later down the line

* when working with CSVs, in my area (ML / scientific compute), Python is often the right context to be in.

[stanleydrew]

> I've experienced many times how highly skilled software engineers groan and sweat when they encounter shell scripts. I don't understand why, but it seems like people with a CS background are never really taught shell scripts and have come to irrationally fear them. It's sort of taboo.

It's not a lack of being "taught" shell scripts. It's the fact that shell programming constructs aren't well documented, your "standard library" is basically dependent on whatever binaries happen to be available on the filesystem, error handling is almost non-existent, etc.

It's very easy to write a bad shell script that "solves" a problem as long as a bunch of assumptions aren't violated. In my experience, senior software engineers are extremely averse to hidden assumptions and very concerned with reliability of the systems they build.

[pixl97]

Yea... "Well the script works fine in MY SYSTEM" was the most common issue with said scripts. Running across different versions of Linux was fraught with issues, much less any other operating system that could execute a shell script.

Of course this can happen with any language, especially as it ages and adds complexity.

[imiric]

I certainly have, and might have written a few of those myself.

But this doesn't make this approach inherently wrong or obsolete. The programmer is wrong for trying to use the tools beyond their capabilities. Where that line is drawn is subjective, as is the concept of maintainability, but if you feel that you're struggling to accomplish something, and that it's becoming a chore to maintain, the path forward is choosing a more capable tool, like a programming language.

[Scubabear68]

I think we are actually mostly in agreement there then.

Perl was invented because the gap from shell to more capable languages was (and is) really big. Languages like Python and Ruby didn’t exist yet, and Perl had a really, really strong sweet spot in text processing.

[ryl00]

> Perl had a really, really strong sweet spot in text processing.

Still does.

[cutler]

Since Ruby took the best bits of Perl what advantage does Perl retain?

[thesuperbigfrog]

>> what advantage does Perl retain?

Ubiquity, speed, and conciseness.

Perl is usually installed by default on Linux and Unix systems. Ruby might be there, it depends.

Perl is faster than Ruby. Ruby has been one of the slower scripting languages. But Ruby has been working on performance improvements in the past few releases. I have not seen any benchmarks of the current Perl versus the current Ruby, so this may have changed.

Perl is more concise than Ruby allowing more functionality for less code.

[cutler]

I don't know about conciseness. Ruby excels here. As for speed it's useful to distinguish between startup and runtime.

[gamache]

I used Perl then Ruby as my main language for almost a decade each. These days, I don't really write Ruby anymore; I moved on to Elixir and never looked back. But I still find myself using Perl on the command line, in contexts where Awk or Sed would also make sense. Ruby never optimized for the one-liner case IMO.

[cutler]

I don't understand your last point. `ruby -e` has excellent parity with `perl -e`.

[moomin]

Yeah, the actual experience of leaky abstractions and non-portable code is forgotten. Perl solved a very real problem in the 90s. Grief, I shudder to think back to the sheer complexity of my bashrc file back then.

[beebeepka]

Yes, exactly. I have seen entire backend systems written in bash. Everything was shell script, sed and awk. The owner didn't want python or perl because he only knew bash and the related tools.

Everything was needlessly hard because these tools were not built for that. Easy to talk about philosophy and the "classic Unix approach" if you don't have to build modern applications this way.

[unpythonic]

Horror is apt.

You hit it on the head with the slowness of loops when the body comprises a series of program invocations. The horror really seeps in when you realize the original author wasn't stopped by the lack of data structures: they could get around that with some creative variable names.

[mulmen]

Programming environments including shells and operating systems are just tools. And every tool can been misused. I opened a can of beans with a screwdriver once. Reality is messy. That doesn’t make the tool bad.