[ZeroGravitas]

A generic (partial) solution to this type of thing is just to sample a number of lines from the large input, and do the investigation on that.

shuf -n 1000 file

This is part of coreutils.

There's also jiq, which is a clone of jid (mentioned elsewhere) but with jq syntax

[dewey]

That would only work if it's line separated JSON though. If you cut off the first 1000 lines of a big JSON file it will be invalid.

[Someone]

Then, use

  jq —-compact-output '.' | head -10 | foo

That is also useful for grepping to filter on records of interest.

jq also has --stream for handling large inputs.