[tester457]

On the topic of xargs replacements, I love gnu parallel.

The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs.

Parallel has an option for almost everything, it's almost too much.

But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0]

The gnu parallel book and reading materials [1] are excellent too.

[0] https://www.gnu.org/software/parallel/parallel_alternatives....

[1] https://www.gnu.org/software/parallel/#Tutorial

[somat]

I have used echo as a sort of poor mans equivalent for a safe check of a pipeline, removing the echo when I felt the rest of the pipeline was working correctly.

  shell stuff | xargs -n 1 -I % echo real command and % args

I have also been known to write scripts where instead of executing the critical parts it prints them. Then a dry run is

  script

and the real run is

  script | sh

[krackers]

Shouldn't you use echo instead of cat?

[somat]

yes echo, total brain fart there. corrected.

[bryancoxwell]

Love that idea, thanks for sharing

[dataflow]

Oh man, parallel is awful. I have to rant about this because I literally tried it again today morning.

Every single damn time I try parallel and decide to give it another chance, something ends up not working or causing a problem. I can never get it to just do what I want and get out of the way.

Today I foolishly thought maybe I was the one who was holding it wrong every single time in the past, so I copy pasted another command that was supposed to work, and thought surely this would be straightforward. Boy was I wrong. I got some manifesto about academic citations and plagiarism, which confused the hell out of me. After I wasted time trying to figure out how to turn off that nonsense, the app just hung there trying to figure out how long its command line can be? Literally doing nothing? What the hell? I killed it but then my terminal didn't close because every time I did this apparently some perl command was spawned in the background blocked on nothing. Why the hell was perl even relevant? Nothing I wrote used Perl. Just run the darn commands I asked in parallel, is that so hard?

[nvme0n1p1]

> the app just hung there trying to figure out how long its command line can be?

That's common on linux. Many tools read from stdin if a file path isn't given: cat, xargs, base64, cksum, etc.

The citation thing is a little silly, I'll give you that one.

[dataflow]

> That's common on linux. Many tools read from stdin if a file path isn't given: cat, xargs, base64, cksum, etc.

No. I did pipe to stdin. It's not my first time using Linux...

Here's a command line I ran right now, and the output I see:

  $ echo "http://www.example.com" | parallel -k -j 8 curl -s "{}"
  Academic tradition requires you to cite works you base your article on.
  [...more nonsense...]
  To silence this citation notice: run 'parallel --citation' once.
  
  parallel: Warning: Finding the maximal command line length. This may take up to 1 minute.

So I wait a few seconds... until I get fed up and look at my process list, and I see perl is just... seemingly sitting there, doing seemingly absolutely nothing. I'm not going to waste a whole minute of my life waiting for this; I see no reason competently written software should take that long just to accomplish such a simple task where nothing is remotely close to reaching any limits.

So I Ctrl+C. And then the parent perl process gets killed, but the child apparently keeps running.

I press Ctrl+D to exit the terminal, and then:

  $ # (Ctrl+D pressed)
  logout

...it just sits there waiting. Ctrl+C and Ctrl+\ do nothing. I have to kill the lingering perl process manually.

xargs Just Works without any of this nonsense, yet somehow I'm the one holding GNU parallel wrong?

[darrenf]

> parallel: Warning: Finding the maximal command line length. This may take up to 1 minute.

> So I wait a few seconds [snip]

This warning is only ever printed if running in Cygwin, not Linux or macOS or elsewhere. Cygwin is notoriously slow.

    # This is slow on Cygwin, so give Cygwin users a warning
    if($^O eq "cygwin" or $^O eq "msys") {
    ::warning("Finding the maximal command line length. ".
          "This may take up to 1 minute.")
    }

Also note that it only figures this out first time, after which it’s cached on disk.

[dataflow]

This is on MSYS2, yes, and that excuses absolutely nothing, because this shouldn't be happening in the first place for speed to be even relevant. At the risk of repeating myself thrice: the messages are confusing, the Ctrl+C handling is just utterly broken, the citation message adds to the confusion while being frankly obnoxious, and all of the delays and outputs are unnecessary in the first place as proven by literally every other program that doesn't make me wait a minute before I can use it the first time, including xargs. If Microsoft's own Windows tools did this, everybody would bash them (no pun intended) till the end of time. But since it's GNU Parallel and not Microsoft Parallel, it's Windows's fault for being slow and also my fault for having the audacity to expect better, apparently.

[MrDOS]

> the messages are confusing > while being frankly obnoxious

Try reading some time. It's pretty great.

> the Ctrl+C handling is just utterly broken

That's your terminal, not parallel.

> and all of the delays

Your terminal is adding the delay, not parallel. parallel is just warning you about your broken terminal. You're shooting the messenger here.

[brokenmachine]

I'm not new to command line tools, but I somehow managed to delete ~10k pictures while trying to use parallel to resize them.

I forget the details but it was some kind of surprisingly weird foot-gun behavior.

Luckily I had a backup, but it really has made me scared to try using parallel again.

[shawn_w]

>Why the hell was perl even relevant? Nothing I wrote used Perl.

parallel is written in perl.

[fn-mote]

> Today I foolishly thought maybe I was the one who was holding it wrong […]

Apparently goes on to describe being confused about parallel reading from the standard input?

[dataflow]

No: https://news.ycombinator.com/item?id=48988369

[Ferret7446]

Are you using the wrong parallel? Moreutils unfortunately ships a (IMO much worse) parallel

[tadfisher]

This is pretty funny. You almost had me until the Perl part. Not going to get baited this time!

[metadat]

Every time I’ve used Gnu Parallel on a new system it required accepting a Eula, which is annoying. I stick to xargs -P99 these days and am happier.

Utility software which has non-essential different first-run behavior is hostile to users.

[quotemstr]

I love GNU Parallel too. I love how it's free software and how I can patch out the obnoxious citation notice.

[sidewndr46]

isn't parallel that tool that dumps some kind of begging message into your output each time you invoke it?

[Ferret7446]

Do you really equate a one time request to cite something if you use it in an academic paper with begging? I know they say that users of open source software are entitled, but this seems rather extreme.

[shawn_w]

No? You can tell it to stop printing the citation notice.

[saghm]

That sounds a lot more like "yes, but..." than "no"

[jeffbee]

Parallel is ok but rush is nicer and much faster.

https://github.com/shenwei356/rush

[foobarqux]

The rush project page on Github itself says it is only "slightly" faster.

[jeffbee]

Modesty, I suppose. The larger your machine is, the more obviously faster it is.