[hn92726819]

> I hate that the jq command is always some indecipherable string in the middle of the script

It might be worthwhile to just learn how jq works. At the end of the day, you need to learn some language to parse json. I hate DSLs too, but I cannot think of anything as useful and concise as jq.

> but that ends up being at least as nasty as the JQ script

That's exaxtly why jq is so nice. Nice alternatives just don't exist

[p-e-w]

> That's exaxtly why jq is so nice. Nice alternatives just don't exist

Write a simple Python script, parse JSON into native objects, manipulate those objects as desired with standard Python code, then serialize back into JSON if necessary. Voila, you have a readable, maintainable, straightforward solution, and the only dependency (the Python interpreter) is already preinstalled on almost every modern system.

Sure, you may need a few more lines of code than what would be possible with a tailor-made DSL like jq, but this isn't code golf. Good code targets humans, not "least possible number of bytes, arranged in the cleverest possible way".

[darkwater]

The simple existence of DSL tools like jq is the testament to the fact that people don't want to go to a generic language to solve every kind of problem. I'm also convinced that a big subset of "use generic language for everything" do it because they want to use their shiny hammer on that nail as well.

[6510]

If I don't use something often enough I just forget how it works.

[hnben]

> Sure, you may need a few more lines of code than ...

jQ integrates very nicely into bash script. Especially in between pipes a short&simple jq-snippet can work wonders for readability of the overall script.

On the other hand, if the bash script becomes too complex it may be a good idea to replace the entire bash script with python (instead of just the json-parsing-part)

[p-e-w]

> a short&simple jq-snippet can work wonders for readability of the overall script.

... if the reader happens to be familiar with the niche language "jq".

Otherwise, you may as well have put some Akkadian cuneiform in there.

[mmcdermott]

> ... if the reader happens to be familiar with the niche language "jq".

Eh. Linux/Unix has always had an affinity for DSLs and mini-languages. If you're willing to work with bash, sed, awk, perl, lex, yacc, bc/dc etc. jq doesn't seem like it should cause too much consternation.

[Ekaros]

So awk or sed or maybe even grep?

jq seems slightly better than those...

[cryptonector]

Certainly for JSON.

[viraptor]

> Especially in between pipes a short&simple jq-snippet

Many of them are not short and simple though. And each time you do a some transformation, you pretty much need to go in/out of jq at each step of it want to make some decisions or get multiple types of results without processing the original multiple times.

[ianleeclark]

The point in my career at which I used jq the most was when I was doing a lot of work with Elasticsearch doing exploratory work on indexed data and search results. Doing things such as trying to figure out what sort of values `key` might have, grabbing ids returned, etc.

Second to this, I've mostly used jq to look at OpenAPI/swagger files, again just doing one-off tasks, such as listing all api routes, listing similarly named schemas, etc.

From what I've seen in the companies I've worked for, this is fairly consistent, but naturally I can't speak for everyone's use-cases. At the end of the day, I don't think most people use jq in places where readable or maintainable would be most appropriate.

[datatrashfire]

Yea except the python solution is probably going to be several hundred lines, instead of a few.

Python is often not installed in server environments unless it's a runtime environment for Python.

Want to use a non standard library? Now your coworkers are suddenly in Python dependency hell. Better hope anyone else that wants to use this is either familiar with the ecosystem, or just happens to have an identical runtime environment as you.

Or someone could just curl/apt/dnf a jq binary to use your 3 line query, instead of maintaining all of this + 200 lines of Python.

[hn92726819]

I got to jq for the same reason I go to regular expressions. If you tell me this is too complex

    (?:[A-Z][a-z]+_?(\d+))

Then I don't know what to tell you. Do you think that's too complex and should be a python script too? I don't think so. It looks complex, but if you just learn it, it's easier than a 'simple' script to do the same thing.

I'd argue it's good code if you don't have to sift through lines of boilerplate to do something so trivial in jq or regex syntax.

[rwmj]

jq is very practical compared to writing python programs. It is the workhorse of many of our tests, eg:

https://gitlab.com/nbdkit/libnbd/-/blob/master/info/info-jso... https://gitlab.com/nbdkit/libnbd/-/blob/master/info/info-map... https://gitlab.com/nbdkit/nbdkit/-/blob/master/tests/test-ex... https://gitlab.com/nbdkit/nbdkit/-/blob/master/tests/test-ta...

(I picked a few completely at random from dozens and dozens)

[wwader]

I do lots of exploratory work in various structure data, in my case often debugging media filea via https://github.com/wader/fq, which mean doing lots of use-once-queries on the command line or REPL. In those cases jq line-friendly and composable syntax and generators really shine.

[comprev]

jq can be easily downloaded, unpacked, executed and deleted in one line of bash.

It's beauty is the simplicity and portability.

[speed_spread]

Something not having alternatives doesn't make it necessarily nice. It's okay to wish for better even if you have something that works.

[hn92726819]

> Something not having alternatives doesn't make it necessarily nice

Of course not, but compared to every alternative today, jq is eons better than everything else. It's conciseness, ease of use, ease of learning all make it awesome. So as of right now, it is the nicest thing to use by far.

Personally though, I don't think I do wish for better. Jq is missing nothing that I want.

[ustolemyname]

I think in this case "better" reduces to convincing the upstream data source to not use json.

Putting that frustration on jq seems like a case of transference.

[Nullabillity]

And now you've turned a JSON traversal problem into a parsing problem.. congratulations?

[spion]

Or maybe jq does have some design flaws.

[jordibc]

I really like jq, but I think there is at least one nice alternative to it: jet [1].

It is also a single executable, written in clojure and fast. Among other niceties, you don't have to learn any DSL in this case -- at least not if you already know clojure!

[1] https://github.com/borkdude/jet

[ahoka]

What about JSONPath?

[mdaniel]

It may be fine for hello-world stuff, but how would one JSONPath this?

  echo '[{"name":"_skip"},{"name":"alpha"},{"name":"_other"}' | \
    jq '[ .[] | select(.name|test("^_.*")|not) | . ]'

The same is roughly true for JMESPath, also, although at least it does actually try to allow projections and some limited functions

[ahoka]

I believe like this.

  $[?(@.name =~ /[^_].*/)]

[mdaniel]

Interesting, I only knew about <https://goessner.net/articles/JsonPath/index.html> which makes no reference to =~ nor any such // syntax but it seems there's an IETF draft that is (a) an actual specification (b) includes more modern stuff like what you used: https://www.ietf.org/archive/id/draft-ietf-jsonpath-base-07....

And, while whatever is powering https://jsonpath.com/ does honor your syntax, albeit with an absolutely useless result:

    [
      {
        "name": -1
      },
      {
        "name": -1
      },
      {
        "name": -1
      }
    ]

I found that `pip install jsonpath-ng` does not accept it nor mention it <https://github.com/h2non/jsonpath-ng/tree/v1.5.3?tab=readme-...> so I think it's out on the bleeding edge or something

[storoj]

Is ".*" necessary? I think "^_" should be enough.

[hn92726819]

I hadn't seen this before. At a quick glance, the syntax looks fine. Though I don't know what command line utility I'd need to use it. It makes me wonder how hard a translator from jq syntax to jsonpath would be... Then we could have our cake and eat it too.