| HN Mirror

[0] https://github.com/jhawthorn/rapidjson-ruby/blob/518818e6768...

> Using the benchmarks in the Oj test directory

I'm sorry, but I've looked for a while now, and I can't seem to identify the benchmark you are mentioning. I suspect it's the one John took for his benchmark suite? [0]

> Oj has a slight advantage over the core json for dumping but not enough to make much difference

I'd be curious to see which benchmark you are using, because on the various ones included in ruby/json, Oj is slightly slower on about all of them: https://gist.github.com/byroot/b13d78e37b5c0ac88031dff763b3b..., except for scanning strings with lots of multi-byte characters, but I have a branch I need to finish that should fix that.

> The comparison for Oj strict parsing compared to the core json is more substantial as 1.37 times faster

Here too I'd be curious to see your benchmark suite because that doesn't match mine: https://gist.github.com/byroot/dd4d4391d45307a47446addeb7774...

> The callback parsers (Saj and Scp) also show a performance advantage as does the most recent Oj::Parser.

Yeah, callback parsing isn't something I plan to support, at least not for now. As for Oj::Parser, `ruby/json` got quite close to it, but then @tenderlove pointed to me that the API I was trying to match wasn't thread safe, hence it wasn't a fair comparison, so now I still bench against it, but with a new instance every time: https://github.com/ruby/json/pull/703.

> You picked the wrong options for you example.

No, I picked them deliberately. That's the sort of behavior users don't expect and can be bitten by. As a matter of fact, I discovered this behavior because one of the benchmark payloads (canada.json) doesn't roundtrip cleanly with Oj's default mode, that's why I benchmark against the `:compat` mode. IMO truncating data for speed isn't an acceptable default config.

The strict mode benchmarks for Oj are in the test/perf_strict.rb. Others are are in perf_*.rb.

If callback parsing is not supported that's fine. Oj does support callback parsing as it allows elements in a JSON to be ignored. That save memory, GC, and performance. Your choice of course just as including callback parsers is a choice for Oj.

Ok, so you picked options that you knew would fail. Again you choice but there are certainly others that would trade a slight improvement in performance to not have 16+ significant digits. It's a choice. You are certainly entitled to you opinion but that doesn't mean everyone will share them.

I'm not sure what platform you are testing on but i'm sure there will be variations depending on the OS and the hardware. I tested on MacOS M1.

> If callback parsing is not supported that's fine.

Yes, as mentioned in part 1 of the series, my goal for ruby/json, given it is part of Ruby's stdlib, is to be good enough so that the vast majority of users don't need to look elsewhere, but it isn't to support every possible use case or to make a specific gem obsolete. For the minority of users that need things like event parsing, they can reach to Oj.

> but that doesn't mean everyone will share them.

Of course. When I was a fairly junior developer, I heard someone say: "Performance should take a backseat to correctness", and that still resonate with me. That's why I wouldn't consider such truncation as a default.

> i'm sure there will be variations depending on the OS and the hardware. I tested on MacOS M1.

I suspect so too. I'd like to get my hands on a x86_64/Linux machine to make sure performance is comparable there, but I haven't come to it yet. All my comparisons for now have been on M3/macOS.

> It looks like a lot of time and effort went into the analysis.

It was roughly two weeks full time, minus some bug fixes and such. I think in the end I'll have spent more time writing the blog series than on the actual project, but that probably says more about my writing skill :p

Anyway, thanks for the pointers, I'll have a look to see if there's some more performance that need to be squeezed.

If you would like to discuss separately on a call or chats I'd be up for that. Maybe kick around a few ideas.

I missed responding to your assertion that the Oj::Parser was not thread safe. An individual Oj::Parser instance is not thread safe just like other Ruby object such as a Hash but multiple Oj::Parser instances can be created in as many threads as desired. The reason each individual Oj::Parser is not thread safe is that it stores the parser state.

Yes that's what I meant. The benchmark suite I took from rapidjson was benchmarking against:

    Oj::Parser.usual.parse(string)

That is what isn't thread safe. And yes you can implement a parser pool, or simply so something like:

   parser = (Thread.current[:my_parser] ||= Oj::Parser.new(:usual))

But that didn't really feel right for a benchmark suite, because of the many different ways you could implement that in a real world app. So it's unclear what the real world overhead would be to make this API usable in a given application.

> is that it stores the parser state.

And also a bunch of parsing caches, which makes it perform very well when parsing the same document over and over, or documents with a similar structure, but not as well when parsing many different documents. But I'll touch on that in a future post when I start talking about the parsing side.

Just so you know, I am impressed by the depth you've delved into with JSON parsing and dumping. It looks like a lot of time and effort went into the analysis.

Ah, I figured why on the Oj side `ruby/json` appeared slower: https://github.com/ohler55/oj/pull/949

Merged. Didn't seem to make much difference though. Results for the original Oj parser are pretty close to the core json now. I'll have to update the README for Oj. It's a bit stale. The new Oj::Parser is still much faster if not restricted to the current Rails environment.