[fbcpck]

Somewhat relevant: the default sed that ships on all macos is freebsd sed.

The basic featureset and syntax is about the same, but the more advanced parameters and features have different syntax (or not implemented) when compared to gnu sed.

This caused me quite a headache in the past figuring out why certain scripts works (or breaks!) on local machine but not in others; I started taking closer look on all tool versions since then.

(The solution is to just use gnu sed with https://formulae.brew.sh/formula/gnu-sed)

[asicsp]

See https://stackoverflow.com/q/24275070 for a list of differences between various sed versions. Just seeing the length of the answer is intimidating.

Perl is another great option for portability.

[NegativeLatency]

Another plus of Perl is that it’s a little easier to specify tabs new lines and other escaped characters than in sed in my experience. In Perl I’ll usually get it right the first time, but in sed I might have to do something dumb like sticking a literal newline in a variable or something.

[tyingq]

Similar for other tools. "Awk" on MacOS is mawk, which is significantly different from the gawk that Linux uses as "awk".

Edit: As mentioned below, awk is mawk for some Linux distros, gawk for others, and something else on still others :)

[pletnes]

True, although sometimes mawk is installed, as others mention.

If you need gawk, put gawk in your shebang, and if nothing else the end user has a chance at understanding why they’re seeing a crash. Also it will crash on line 1, nice and loud.

[version_five]

Which linux are you referring to? I'm almost certain than Ubuntu 16.04 and 18.04 both ship with mawk as the default.

Edit: just checked and my vps than runs 20.10 uses gawk

Edit 2: 18.04 also comes with gawk, at least the install I have. I know I have worked with mawk on a linux machine- could it have been Ubuntu 16.04?

[spinax]

IIRC (no Ubuntu VM handy right now) mawk is the default with a minimal LTS install, using alternatives to point awk at it. When you apt install gawk it has a higher priority in alternatives and flips the awk symlinks over to gawk automagically.

[version_five]

> gawk it has a higher priority in alternatives and flips the awk symlinks over to gawk automagically

That must be what happened. I just checked my other machine that had a pretty new install of 20.04 on it, and it does have mawk. I'm happy to see I'm not crazy.

[tyingq]

Ah...yeah, I was probably over generalizing when I said that, though it's probably fair to say that gawk is the default awk for many popular Linux distros.

[shakna]

Debian & Ubuntu have moved back and forth between using mawk or gawk several times.

Debian Jessie (circa 2016) used gawk.

Debian Stretch (circa 2017) used mawk.

Today, Debian uses gawk.

[JNRowe]

Have you flipped the final one? Both my Buster and Bullseye installation only have mawk.

Trying to dig in deeper makes my head explode though. mawk is marked required yet with update-alternatives priority of 5, and gawk is marked optional but with a priority 10. I guess the intention is that mawk is good enough for base installs, but you probably want gawk.

[tyingq]

It's definitely hard to track down. Mawk is often way faster than gawk, but gawk has more features (sockets even...crazy) and maybe fewer arbitrary limits.

[CraigJPerry]

There's a bunch of tools that trip me up between the BSD and GNU versions.

Find is the big one - so much so that i "cargo install fd-find" on my boxes these days and use the rust version across all hosts.

I briefly flirted with using zsh's enhanced glob abilities, e.g.

  find . -type f -size +1m -ls

vs.

  ls **/*(.Lm+1)

[remram]

The biggest problem with this is using the -i flag ("in place") which expects an argument on BSD/MacOS but not on GNU/Linux.

[1vuio0pswjnm7]

"The solution is to just use gnu sed with https://formulae.brew.sh/formula/gnu-sed)"

Problem for me is I use more than just MacOS and Linux. I would have install GNU sed on every OS I use. I make small OS images for booting from USB. I use a non-busybox multi-call binary as the initial userland to conserve space, part of filesystem embedded in the kernel (sort of like initrd on Linux I guess). Early in the boot and setup process I use sed in scripts. Thus I would have to make sure there was a copy of GNU sed installed, preferably added to the multi-call binary. Not sure the work is really worth it just for a few convenience features. I can do anything I need to do with BSD sed just as well as with GNU sed.

Instead, the solution I chose is to write sed scripts for NetBSD's sed. These work on Linux, MacOS, FreeBSD, OpenBSD, Plan9, etc., old or new. There is one feature it has that GNU does not: "-a"; it can be used to do true (but limited) "in-place editing" without creating a temp file. Not sure why but NetBSD has since changed their sed to be more FreeBSD and GNU-compatible, e.g., adding "-i" and "-g/-G", to enable automatic temp file creation then removal^1 and GNU regex, respectively. However if we start using these "features" everywhere, then our scripts may not work with older userlands.

1. This is often called "in-place" editing, but if one examines the source code one will see there is still a temp file (tmpfname) created. We are being spared the step of manually creating then removing a temp file, but we still need the requisite filesystem space available for one.