[cosmiccatnap]

I would love to see what jq looks like on something like a 1mil line Json vs this. In my experience jq syntax is fine and I've not ran into a performance issue on any one file but I seem to see a jq clone every few months on here so someone seems to need that, or maybe it's just the new volume slider problem who knows.

[justinsaccount]

jq performance is pretty terrible. Here I'm going to do something super simple like pull out a single field out of a large log file:

  $ wc -l big.log 
    979400 big.log

  $ du -hs big.log 
  570M big.log

`count` is a small program that counts lines on stdin. like `sort|uniq -c |sort -n`

jq takes 12 seconds:

  $ time cat big.log |jq -cr .method |~/bin/count 
  848000 GET
  94800 POST
  34000 HEAD
  2400 OPTIONS
  200 null

  real 0m12.381s
  user 0m12.427s
  sys 0m0.333s

my tool takes .5 seconds

  $ time cat big.log |~/bin/jj method |~/bin/count 
  848000 GET
  94800 POST
  34000 HEAD
  2400 OPTIONS
  200 

  real 0m0.466s
  user 0m0.512s
  sys 0m0.198s

`jj` is a little tool I wrote that uses https://github.com/buger/jsonparser