[optionalsquid]

> In it, I learned that there's some interesting quirks about the way the traditional shaXsum tools work, specifically that they focus on a set of inputs from the shell command line, which means that you can exhaust the shell command line length fairly easily (try hashing every file in the Linux source tree, for instance; zsh, bash, and fish will all eventually tell you "Fuck off, there's too many characters in argv")

Did you try to run something like

  sha256sum **/*.*

Because if so, then you'll of course exceed the maximum number of arguments. That applies to all command-line tools that take an unbounded number of arguments.

A simple solution is to use find, which handles that for you:

  find -type f -exec sha256sum "{}" "+"  # 1.4s on my system for Linux 7.1.5

Alternatively, you can combine find with xargs (or parallel), which also handles it and optionally allows you to specify exactly how many arguments to process per call (-n), and how many commands to run in parallel (-P):

  find -type f -print0 | xargs --null -P 8 -n 512 sha256sum  # 0.3s