[Ambol]

These type of things is why I always use a real programming language rather than shell scripting. Can't wrap my head around all the different pitfalls of treating random string outputs as input for other commands without first validating and parsing them into an actual data structure.

[as-j]

Shell scripting has it’s place. In general it works fairly well and it’s an important skill to have.

I used to have a report who had a similar train of thought, and ended up writing shell scripts in python which was way more fragile, unreadable and more complicated.

The edge cases are there, but you don’t run into them all that often. Or if you do, you fix your scripts. Take the cp —- 2nd example. You generally don’t start file names with a -. It just makes Unix life annoying so you get pushed away from doing it.

Keep scripts sane and straight forward, you can do really nice and powerful work knowing Unix tools. You can also shoot yourself in the foot some day when you create a file called ‘-rf’.

[3pt14159]

Bash is like a spec ops soldier's knife. Sure it could be used to kill someone (Bash on Balls, anyone?), but it's mostly for doing boring stuff like opening MREs (getting around a filesystem) or removing a deep thorn from the skin (removing a stupid "$n/a" string in a one-use TSV).

I wouldn't send a navy seal into battle without one, but it isn't an absolutely critical piece of gear.

[jeremysalwen]

Many people have responded to your comment saying that this is bad and that using other languages makes things more complicated when you are doing 'bash-like' things. I'm guessing they haven't actually tried it, because I have used python with the [sh](https://github.com/amoffat/sh) package many times to replace bash scripts, and I have never regretted it.

[jayd16]

I don't like having to deal with the external dependency of python being on whatever machine or container my script runs on.

[samatman]

This was where perl really shined.

Still can, if you want it to. I'll admit my perl chops have rusted clean away.

But it used to be something you could rely on having, and never had the dependency hell problem that's plagued python, for whatever reason.

Also never went through the 2-to-3 transition tsuris, instead opting to make its sequel the Duke Nukem Forever of programming languages.

[cle]

I definitely relate with this. This is why I do my “scripting” these days in Go. Programs aren’t editable like Python, but compiling a Go program results in a single executable that I can just drop onto a host somewhere, without worrying about interpreter versions and pip and all that crap.

I haven’t found Bash to be much better in practice. You can’t really do much with Bash alone, it’s only as useful as the tools you use it with, the ones you install with your OS package manager. And Bash versions themselves can vary dramatically. Try writing non-trivial Bash scripts that are portable between macOS and Linux...it’s a nightmare.

(Yes I know there are libs for python that can do this, but in general Go is just simpler: it has everything I need for writing utilities out of the box, including way simpler package management, and I can come back 2 years later to some code I wrote and jump back into it with no effort.)

[arminiusreturns]

>Bash alone, it’s only as useful as the tools you use it with

This is like half the point for me. It's that I have lots of cli tools I like to use, and many of them are very unix-philosophy like. I want to cobble them together to accomplish a task, and since they already work with shell and pipes etc, it's trivial.

Now in an alternative, I have to go find some library that does the equivalent, or write my own version of the functions, etc, and lots of times those libraries have the thier own sets of pitfalls. I like bash because it encourages sticking the unix philsophy of do one thing and do it well (because you are just using it as a glue, not as a material itself). (also sometimes I just want something thats easy to read and understand)

For all those who bash bash as so horrible, unstable, non-prod.... bash scripts aren't perfect, and nobody who uses bash would say so, but bash runs on millions of prod systems across the world and things are fine... so maybe whats needed is a bit less holier-than-thou attitude from those people and bit more "you do you".

[cle]

The point is that you still have to make sure those tools are installed on the system, and compatible with the code you wrote. Standard CLI programs we rely on in our Bash scripts can vary dramatically between Linux distributions and OSs. If you want portable Bash code you need to bundle it in a container w/ all of its dependencies, or be very judicious about what programs and program features you use.

I'm not bashing Bash, I use Bash all the time and it's enormously useful. But it has limitations and tradeoffs that make me reach for other things more often.

[datashaman]

One of the beauties of bash is that it's _not_ compiled.

Can you read the Go source code from a Go binary? Is the algorithm transparent to the user?

How do you jump back to work on a binary? You have to keep the source code somewhere else (probably in a repo somewhere); now you have an application instead of a script.

[tikkabhuna]

> You have to keep the source code somewhere else (probably in a repo somewhere)

You should be doing that for scripts as well! Far too often I've seen problems arise when a team depends on a process but its just a shell script run by cron from the home directory of the guy who's on vacation.

You could also do `go run main.go` if you want to be close to the source. The Gitlab Runner repo has a "script" run like that[1].

[1] https://gitlab.com/gitlab-org/gitlab-runner/-/blob/master/Ma...

[cle]

Yeah that's one of the first things I mentioned. Go executables aren't editable. In practice it barely makes a difference, if you can SSH onto a machine then you can SCP a binary onto it too. At that point you edit the source code and press Ctrl+P, Return to rerun "go build -o fix-the-thing main.go && scp fix-the-thing $HOST:/tmp && ssh /tmp/fix-the-thing". And then you've got your code ready to check in to source.

[nijave]

Yup and something rarely mentioned is lack of good testing frameworks and all the pitfalls when you try to make something portable

I will admit Python doesn't have great, simple primitives for shell-like tasks but you can create a few functions to make a simple sh("") wrapper.

With Ruby's backtick execution syntax you can blend in shell/system utilities pretty easily. Unfortunately, Ruby isn't installed on as many systems by default, afaik

Here's a Ruby equivalent of looping through mp3 files in the current directory

```

Dir["*.mp3"].each do |file|

  puts file # print the file name

end

```

[bluedino]

I find a lot of times someone extends or duplicates a bash script and soon gets over their head in complexity, or doesn’t realize how much easier a Ruby/Python/whatever script would be to change, mostly because it’s so much more readable.

Bash has all kinds of pitfalls and things like checking if a file exists vs if it’s larger than 0 bytes.

Then you get people who instead use php to do scripts. Ugh.

[rantwasp]

yeah no. you should start with bash and drop into a proper programming language when complexity warrants it

[jayd16]

What kind of parsing do you think can be done in "a real programming language" that can't be done in bash?

I like objects and performance and abstraction and other things about other languages but bash has a _lot_ of parsing tools.

[ryl00]

This is where perl really shines for me. Regular expressions are front and center, making it so frictionless to slice and dice text. My bash knowledge is so stunted because I bail to perl as soon as the going gets tough. :)

[dredmorbius]

Bash's regex library is called "sed".

[hnlmorg]

`sed` differs from different implementations (eg GNU vs BSD).

Bash does actually have a fair amount of regex support built in too though it's not as well talked about.

[dredmorbius]

Both points taken.

Mine remains: that shell affords capabilities via utilities. Awk (again, with numerous implementations) also offers regex capabilities, some not offerred by sed, and missing a few.

When writing cross-platform scripting, adhering to common standards rises in importance. This is possible, if occasionally limiting.

[renewiltord]

What frameworks/libraries do you use? I find that the reason I prefer using a shell script is:

* ergonomic integration with the shell

* easy piping

* easy input/output redirection

Maybe Ruby comes close with the backticks or `system` but it still isn't as nice as a shell IMHO

[GuB-42]

For me, it would be Perl.

Perl is influenced by shell scripting and there are several easy ways to run commands, build pipes, redirect and other things shells are good at, like testing and listing files.

And as ugly as Perl looks, it has much less pitfalls than shell scripts.

[renewiltord]

I wrote some Perl code once and it looks like you need more than a passing familiarity with the language to create code that is not just write-once.

[GuB-42]

With proper development practices, you can write clean Perl, even if you don't know much about the language. It has the same constructs as most other procedural languages and you can apply the same principles. Object orientation is a bit tricky though.

The thing is, Perl won't help you with discipline. If you want write-only code, Perl will compile it, no problem.

For that reason, it is the language of choice for one liners and throwaway scripts (and that's how I use it most or the times). But writing clean Perl is perfectly doable, though I usually prefer static languages if maintainable code is a priority (no Perl, no shell, not even Python).

[im3w1l]

While bash itself has some warts, many of the issues are inherent to spawning child processes for accomplishing tasks. Thus, when possible, I try to use proper libraries rather than invoking commands from any language at all.

[renewiltord]

I see. Which ones have you found most ergonomic to use?

[im3w1l]

Python and as for libraries, well that's very dependent on what I want to accomplish, obviously.

[Spivak]

The cost you pay for this is verbose and unergonomic interop when your problem domain is executing a bunch of commands. For all its warts shells shine at being extremely expressive.

It’s usually not worth the effort to shove everything into a real programming language but write a few self-contained commands designed to be called from a shell and then use the shell as the glue.