[hoshsadiq]

I tend to use jq a lot. As others have said, sometimes jq can be hard to grasp. Often it requires multiple attempts to get the correct answer. To make it a little easier for me, I've written a helper function[0] that combines it with fzf[1] to run jq as a REPL on any json. It allows to incrementally alter your DSL without having to continually call jq. This is similar to jid/jiq but a little more powerful. It includes functions to change the preview to output raw, compact (or not), and some other things.

It is essentially similar to jqplay but local.

I didn't use jid/jiq because jid uses go-simplejson, which is nowhere near as powerful as jq, and jiq seemed very buggy when I used it and it felt like it was hacked together. Plus there was no where to change jq's arguments while running it.

I'm sure this function can be improved on, but this has been good enough for me so far.

Also, I run gojq[2] instead of jq. It is a drop-in replacement for jq but is written in Go, and has some improvements over jq such as bug fixes, support for yaml input, and it also provides more helpful error messages.

[0] https://github.com/hoshsadiq/dot_files/blob/master/zshrc.d/m...

[1] https://github.com/junegunn/fzf

[2] https://github.com/itchyny/gojq/

[fisxoj]

If anyone is an emacs user and this sounds compelling, I recommend counsel-jq[0] for the sort of feedback loop described here.

[0]: https://github.com/200ok-ch/counsel-jq

[forty]

I use counsel-jq occasionally, and my main issue with it is the single line input. As soon as the filtering is not trivial, it's much more convenient to be able to use multiple lines.

[psacawa]

I've written about this before, but for tmux users, I have interactive querying the form of ten-line shell script [1]. See it live [2]. It can easily be modfied to interactively query yaml, html with xpath or css, or text with awk or what have, and I have variants for each of these. This has the advantage over the parent comment thatyou have your text editor's key bindings instead of those of fzf.

Dependencies: tmux, nodemon, less, jq, vim

[1]: https://gist.github.com/psacawa/e63c4e25a8b0405309d3a03b6b50...

[2]: https://streamable.com/jwdrqu

[kbd]

Just sharing my take on that interactive jq (or anything else) repl:

https://github.com/kbd/setup/blob/master/HOME/bin/fzr

It's just an fzf wrapper that sets up temporary files and so on. It works really well; it's amazing all the things one can use fzf for.

[ossusermivami]

This is nice, it uses latest/modern python3 features, thanks, I needed some use case to see how those feature are useful in real world...

[alerighi]

I'm the opposite, I find that jq doesn't have a reason to exist, other than pretty printing JSON files on a terminal and doing basic filtering on a JSON object and only as interactive shell usage, NOT in a script.

It's the classical tool, like sed, like awk, like a ton of unix utility that at first they seem to you easy to use, then you have to do something complex and you start abusing them, by piping things multiple times into jq, and you end up writing things like this:

    echo $json | jq "something $(echo $variable | jq 'something else' | tr '"' '\'') | sed 's/"/\\'/g" | jq "another js invocation" | awk '...' > file2.json

I stopped using jq after realizing that I was wasting my time by trying to fix a script that used jq and didn't managed quoting correctly, trying to use different kind of quotes, even filtering the input before passing it to jq with tr replacing things. It's just another tool prone to abuse like sed, awk, tr, cut or similar things.

I thought why I'm wasting my time on a tool that has a complex and limited DSL when I can write a clean python script in 10 minutes to do the same things that is easier to write, to read and most importantly to maintain.

To me a script that has to manipulate JSON should be written in an high level programming language like Python, and not be abused with tool as jq and stuff. Even I there is an already existent big bash script that you don't want to rewrite and you have to do some json processing in it... you can write an inline python script like this:

     python3 <<PYEND
     ## your python code
     PYEND

Also jq is another dependency to a script that must be installed.

[ris]

Speaking as a long-time python developer...

If you actually "get into" jq you find out that it's a significantly neater language than it appears on the surface. Firstly it does allow you write multi-line scripts, and things start to look a lot neater once you do. Secondly it's actually a real, working, functional programming language, which allows very succinct expression of ideas which, in python, would likely require the reader to track state across explicit loops and the like.

Once you dig into the manual, you also tend to discover that a lot of the things that cause you to string multiple jq invocations together aren't actually necessary because there are quite sensible ways of handling them in-language.

It's quite laughable though to tout python over jq because of it adding a dependency. Perhaps if you're already embedded in python-land and all your environments already have python - but many (most?) of us are increasingly targeting extremely minimal container image environments. In that case, adding python is a much larger and more complex dependency than jq's single 3.8MB binary.

As someone who has to read an awful lot of other peoples deployment scripts, it's also quite nice when I see jq because it loudly advertises "all I'm doing here is mangling one piece of json into another! no side effects!". I'd much rather follow the thread of execution into that than some mystery ruby script any day.

[alerighi]

> Secondly it's actually a real, working, functional programming language, which allows very succinct expression of ideas which, in python, would likely require the reader to track state across explicit loops and the like.

This is the problem. Is yet another language that someone has to learn and know, like the ton of other UNIX commands that have their own DSL.

> Once you dig into the manual, you also tend to discover that a lot of the things that cause you to string multiple jq invocations together aren't actually necessary because there are quite sensible ways of handling them in-language.

Nice. I can dig into the manual and spend a day to learn it, but I don't have that time. I have a script to fix, I know how to program in python, and throw away the jq code and substitute it with 10 lines of python in a minute, and problems solved.

> It's quite laughable though to tout python over jq because of it adding a dependency

If jq enters your codebase then everywhere you have to install it. It's not that difficult (but not trivial on Windows), but it is annoying, you run a script and you then find out that you don't have jq installed and you need to install it.

Also what is faster: eliminating the need for the jq command in a script, or installing jq on tens of different systems with different operating systems?

No, where I work I established a rule that every script should be written in python and should use only the standard library, with a few exception (e.g. we work with AWS so boto3 is an exception). If not every developer did use whatever tool they thought it was cool (like jq), write a ton of bash spaghetti code with all these tools used together, and forced every other developer to install them on their system (and cause a lot of work to the IT, i.e. me to do so if they weren't able and fix all the problem).

> Perhaps if you're already embedded in python-land and all your environments already have python

Python is everywhere. Every Linux distribution have a python interpreter in them, same for macOS, and in Windows nowadays you install directly from the store with one click. The problem is that if jq enters the codebase then of course every developer machine has to have it installed. This is annoying.

[cwalv]

I sympathize with both points of view. I think of autotools as an extreme example of how less general tools tend to grow in complexity to the point that the effort to learn them outweighs the benefit you get from knowing them well. But there is something missing from the "just use python everywhere" argument as well. For many usecases, choosing a less expressive language can guard against excessive complexity creeping in at that layer. IMO, json+jq fills a useful niche between semistructured/unstructured-text+awk/sed/bash and APIs+some-gp-scripting-language

[Arnavion]

If you have quoting issues, and going by the example you posted of replacing quotes with sed and tr, you haven't learned how to pass parameters to jq correctly. You should use `--arg` and `--argjson`, not shell string interpolation, to pass in external strings / JSON object strings to your jq command. And use `--raw-output` to convert JSON strings to displayable strings (ie no quotes, evaluated escapes) for output.

[philsnow]

I do the same thing (want to iterate on some complicated jq query -- or, let's be real, blunder around with my loose grasp of jq). I use https://github.com/lotabout/skim#interactive-mode instead of fzf.

  function jqsk {
    sk --tac --ansi --regex  --query . --multi --interactive --cmd '{}' \
       --bind 'enter:select-all+accept,ctrl-y:select-all+execute-silent(for line in {+}; do echo $line; done | pbcopy)+deselect-all' \
       --cmd-history=${HOME}/.sk_history --cmd-history-size=100000 \
       --no-clear-if-empty \
       --cmd-query "cat $1 | jq --raw-output --color-output --exit-status '.'"
  }

I run it like `jqsk file.json` and then change the `'.'` in the cmd-query to whatever I'm trying to do with jq.

I'll definitely look into some of the other mentioned solutions for this though. I settled on skim a year+ ago and haven't revisited.

[asicsp]

Another option for interactive version is: https://sr.ht/~gpanders/ijq/

[rodorgas]

That’s cool! What’s the use of FZF? Isn’t it a fuzzy finder, what are you searching for in a jq REPL?

[hoshsadiq]

It only uses the FZF's preview. The suggestions is completely empty. I tried to find an alternative as FZF has no way of disabling the selector window, but I was unable to find anything that was good enough for this.

I considered forking jid/jiq and using gojq as a library, but I ended up not going down that route because of reasons that I cannot remember. I also considered using a tui or something but FZF has so much already implemented and has a lot of it right, and I didn't particularly feel like re-inventing the wheel.

[rattray]

This looks cool - would you be willing to share it in a brew-installable format?

[hoshsadiq]

Not particularly. I don't use a Mac, but I'd be happy to separate out the function into it's own script so it can be downloaded and put in your $PATH. I personally use zinit to manage individual files from random repos.

[alex_smart]

Why not just use inotify (or similar tools offered by other OS)?