[lysium]

Do you recommend any good alternative when your shell program gets too large?

Honest question, as I’m struggling to leave the shell environment once the program gets too large. I could use Perl, but $? and the likes get quickly out of hand. Python’s support for pipes was difficult last time I used it, but that may have changed. What would you recommend?

[BoppreH]

If it's too large, then just write normal Python code. It'll be a lot longer than the equivalent shell-like script, but you'll gain it back in maintenance effort, debugabillity, and robustness.

[HeckFeck]

You've some hesitation with Perl, but if you stick at it, you'll find what you seek. It feels very 'unixy' and can achieve much the same as shell while being more consistent in its syntax. Its portability means it will work the same across environments. Plus the newest editions have niceties like modern classes and try/catch as inbuilt language features.

Sharing this because its the route I went, anything I'd have written in Bash I'd now do in Perl.

[lysium]

Thank you for encouraging me to use Perl. After Perl 6 came out I got confused at what and how to use and hence I’ve abandoned that path. I’ll try once more now.

[phone[phone]]

The tooling around Perl has also gotten better over the last decade or so while also allowing you to pack everything to run on even ancient machines running old Perl 5.

[cb321]

On Unix, you might try Nim (https://nim-lang.org) with https://github.com/Vindaar/shell and there are a slew of pipeline-y/Unix-y utilities over at https://github.com/c-blake/bu

Nim is statically typed and (generally) native-compiled, but it has very low ceremony ergonomics and a powerful compile-time macro/template system as well as user-defined operators (e.g., you can use `+-` to make a constructor for uncertain values so that `9 +- 2` builds a typed object as in https://github.com/SciNim/Measuremancer .

[lysium]

Thanks for pointing me to nim, it looks promising. I‘ll try to use https://nim-lang.org/docs/osproc.html to pipe programs.

My use case is approx. like this: I can get 80% what I want with ls … | sed … | grep -v … but then it gets complicated in the script and I’d like to replace the sed or grep part with some program.

[cb321]

This sounds like a job for what standard C calls "popen". You can do

    import posix; for line in popen("ls", "r").lines: echo line

in Nim, though you obviously need to replace `echo line` with other desired processing and learn how to do that.

You might also want to consider `rp` which is a program generator-compiler-runner along the lines of `awk` but with all the code just Nim snippets interpolated into a program template: https://github.com/c-blake/bu/blob/main/doc/rp.md . E.g.:

    $ ls -l | rp -pimport\ stats -bvar\ r:RunningStat -wnf\>4 r.push\ 4.f -eecho\ r
    RunningStat(
      number of probes: 26
      max: 31303.0
      min: 23.0
      sum: 84738.0
      mean: 3259.153846153846
      std deviation: 6393.116633069013
    )

[lysium]

Thank you! Popen looks like what I was looking for!

[cb321]

Sure. No problem.

While it may not be in any ANSI/ISO spec for C, even Windows has popen these days. There are also some tiny Nim popenr/popenw wrappers in https://github.com/c-blake/cligen/blob/master/cligen/osUt.ni... covering the Windows case.

Depending upon how balanced work is on either side of the pipe, you usually can even get parallel speed-up on multicore with almost no work. For example, there is no need to use quote-escaped CSV parsing libraries when you just read from a popen()d translator program producing an easier format: https://github.com/c-blake/nio/blob/main/utils/c2tsv.nim

[mplewis]

I use Go. You can run scripts with go run directly, and this package makes shell tasks easy: https://github.com/bitfield/script

[lysium]

Script looks promising, thank you! I’ll give it a try, as some sister comment also suggests Go.

[tesseract]

Good ergonomics for Perl-style quick and dirty text processing were part of the original design goals for Ruby. Those parts of the language are still there. You can write code that feels more concise than Python yet, IMO, tends to be more readable/maintainable than Perl can stereotypically be. Modern style guides, however, de-emphasize that style of Ruby since it might not be the most appropriate in the context of say a large Rails project.

[colejohnson66]

Powershell uses proper objects instead of stringly nonsense.

[reddit_clone]

Perl6/Raku is my personal choice (when I can).

Python to me, is too far away from shell/unix. It is a programming language for writing applications. For the use case of writing shell scripts but in a more powerful language, perl is still the king here (or it should be. Sadly it doesn't appear to be the case. No one is using it except for die hard gray beards.)

Raku is a modern (still a big) language with kitchen sink. Again doesn't appear to be much uptake.

[paddim8]

Elk is a Shell language with syntax similar to Python. https://elk.strct.net

[lysium]

This looks very interesting! Thank you!

[fho]

Unpopular opinion, but I used Haskell "scripts" with relative success for a while. Stack has a nice script interpreter mode that is runnable in the familiar #! way.

Even allows to add dependencies and if necessary compile the script on the fly.

[mongol]

I once read a HN thread that recommended Go for this, and it made me interested. I think it was a useful suggestion, it made me learn Go, and I kind of agree with it, 5+ years after. It is not a shell, but it is simple and fast and useful.

[paulddraper]

Python.

Just the inclusion of argparse alone is worth it IMO.

> Python’s support for pipes was difficult

Well, the idea would be to replace a lot of your pipe usage.

Off the wall, but Scala has a concise syntax for process operations, but startup time is likely prohibitive.