[necheffa]

If you just want to move some files around and do basic text substitution, turning to Python or another other "full fledged programming language" is a mistake. There is so much boiler plate involved just to do something simple like rename a file.

[maccard]

You mean

    import os
    os.rename(“src.txt”, “dest.txt”)

?

[necheffa]

Yes. And it is only downhill from there.

Now, show us `mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log` in Python.

[gorgoiler]

That looks like a snippet from a command session which is a perfectly great place to be using sh syntax.

If it became unwieldy you’d turn it into a script:

  #!/bin/sh

  beautify() {
    sed -e ‘
      s/ugly/beautiful/g
      …other stuff
    ‘
  }

  select() {
    awk ‘
      {print $2, $4}
      …other stuff
    ‘
  }

  mycommand | beautify | select

For me, now it’s starting to look like it could be safer to do these things in a real language.

[necheffa]

I have a lot of scripts that started as me automating/documenting a manual process I would have executed interactively. The script format is more amenable to putting up guardrails. A few even did get complex enough that I either rewrote them from the ground up or translated them to a different language.

For me, the "line in the sand" is not so much whether something is "safer" in a different language. I often find this to be a bit of a straw-man that stands in for skill issues - though I won't argue that shell does have a deceptively higher barrier to entry. For me, it is whether or not I find myself wanting to write a more robust test suite, since that might be easier to accomplish with Ginkgo or pytest or `#include <yourFavorateTestLibrary.h>`.

[xelamonster]

Is it really so bad? A bit more verbose but also more readable, can be plenty short and sweet for me. I probably wouldn't even choose Python here myself and it's the kind of thing shell scripting is tailor-made for, but I'd at least be more comfortable maintaining or extending this version over that:

  from subprocess import Popen, PIPE

  CMD = ("printf", "x:hello:67:ugly!\nyy$:bye:5:ugly.\n")
  OUT = "something.report"
  ERR = "err.log"

  def beautify(str_bytes):
      return str_bytes.decode().replace("ugly", "beautiful")

  def filter(str, \*index):
      parts = str.split(":")
      return " ".join([parts[i-1] for i in index])

  with open(OUT, "w") as out, open(ERR, "w") as err:
      proc = Popen(CMD, stdout=PIPE, stderr=err)
      for line_bytes in proc.stdout:
        out.write(filter(beautify(line_bytes), 2, 4))

I would agree though if this is a one-off need where you have a specific dataset to chop up and aren't concerned with recreating or tweaking the process bash can likely get it done faster.

Edit: this is proving very difficult to format on mobile, sorry if it's not perfect.

[cevn]

In ruby you can just call out to the shell with backticks.

Like.

    myvar = `mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log`

That way, if something is easier in Ruby you do it in ruby, if something is easier in shell, you can just pull its output into a variable.. I avoid 99% of shell scripting this way.

[necheffa]

That is fair...

But if all I need to do is generate the report I proposed...why would I embed that in a Ruby script (or a Python script, or a Perl script, etc.) when I could just use a bash script?

[lostdog]

Bash scripts tend to grow to check on file presence, conditionally run commands based on the results of other commands, or loop through arrays. When it is a nice pipelined command, yes, bash is simpler, but once the script grows to have conditions, loops, and non-string data types, bash drifts into unreadability.

[maccard]

I don’t think it’s fair to compare a workflow that is designed for sed/awk. It’s about 10 lines of python to run my command and capture stdout/stderr - the benefit of which is that I can actually read it. What happens if you want to retry a line if it fails?

[necheffa]

> I don’t think it’s fair to compare a workflow that is designed for sed/awk.

If your position is that we should not be writing bash but instead Python, then yes, it is absolutely fair.

> the benefit of which is that I can actually read it.

And you couldn't read the command pipeline I put together?

> What happens if you want to retry a line if it fails?

Put the thing you want to do in a function, execute it on a line, if the sub-shell returns a failure status, execute it again. It isn't like bash does not have if-statements or while-loops.

[maccard]

My point is that if you take a snippet designed to be terse in bash, it’s an unfair advantage to bash. There are dozens of countless examples in python which will show the opposite

> And you couldn't read the command pipeline I put together?

It took me multiple goes, but the equivalent in python I can understand in one go.

> Put the thing you want to do in a function, execute it on a line, if the sub-shell returns a failure status, execute it again. It isn't like bash does not have if-statements or while-loops.

But when you do that, it all of a sudden looks a lot more like the python code

[pletnes]

Just ask chatgpt and you’ll get a script, probably makes some tests too if you ask for it.

[necheffa]

I have not really been a fan of ChatGPT quality. But even if that were not an issue, it is kinda hard to ask ChatGPT to write a script and a test suite for something that falls under export control and/or ITAR, or even just plain old commercial restrictions.

[murphy214]

import os

os.system('mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log')

[necheffa]

You forgot to point out all those "footguns" you avoided by writing in Python rather than bash...

[pxc]

This has all of the purported problems of doing this directly in a shell language and zero advantages...