[ducaale]

Apparently, the author wrote this tool because jid was struggling with a 7MB JSON file.

See https://github.com/simeji/jid/issues/66#issuecomment-4436718...

[exhaze]

Just tried out both of these for a large endpoint.

- FX "expand/collapse" functionality seems way better for exploring APIs whose shape you don't know

- jid is maybe marginally better for APIs where you have instant recall of the exact shape and need to rapidly query it

Overall, I like FX better because it provides feedback on your query faster.

I am grateful to the author(s) for creating it and I'll be using it instead of JQ whenever I need to wrangle APIs from the CLI.

[ducaale]

FYI, fx has a jid-like mode that can be started by pressing the dot key. See this post[1] for more information.

[1] https://medium.com/@antonmedv/discover-how-to-use-fx-effecti...

[exhaze]

Thank you! I had discovered it when I was playing around with it but forgot to mention that.

[ZeroGravitas]

A generic (partial) solution to this type of thing is just to sample a number of lines from the large input, and do the investigation on that.

shuf -n 1000 file

This is part of coreutils.

There's also jiq, which is a clone of jid (mentioned elsewhere) but with jq syntax

[dewey]

That would only work if it's line separated JSON though. If you cut off the first 1000 lines of a big JSON file it will be invalid.

[Someone]

Then, use

  jq —-compact-output '.' | head -10 | foo

That is also useful for grepping to filter on records of interest.

jq also has --stream for handling large inputs.