Hacker News new | ask | show | jobs
by jnmatsynadia 297 days ago
As the person in question I feel compelled to answer to this: first of all my apologies if I managed to piss you off, certainly didn't mean to!

It looks like you got frustrated by my refusing to give figures of performance for JetStream: I always say in meetings that because there are too many factors that affect greatly JetStream performance (especially compared to Core NATS which mostly just depends on the network I/O) I can not just give any number as that would likely not accurately reflect (better or worse!) the number that you would actually see in your own usage. And that rather you should use the built-in `nats bench` tool to measure the performance for yourself for your kind of traffic requirements and usage patterns, in your target deployment environment and HA requirements.

On top of that, the performance of the software itself is still evolving as we release new versions that improve things and introduce new features (e.g. JetStream publication batches, batched direct gets) that greatly improve some performance numbers.

I assure you that I just don't want to give anyone some number and then you try it for yourself and you can't match those numbers, nothing more! We literally want you to measure the performance for youself rather than to give you some large number. And that's also why the docs don't have any JetStream performance numbers. There is no attempt at any kind of disingenuity, marketing, or pulling wool over anyone's eyes.

And I would never ever claim that JetStream yields the same performance numbers as Core NATS, that's impossible! JetStream does a lot more and involves a lot more I/O than Core NATS.

However, if I get pressed for numbers in a meeting: I do know the orders of magnitude that NATS and JS operate at, and I will even be willing to say with some confidence that Core NATS performance numbers are pretty much always going to be up to the 'millions of messages per second'. But I will remain very resistant to making any claim any specific JS performance numbers because in the end the answer are 'it depends' and 'how long is a piece of string' and you can scale JetStream throughput horizontally using more streams just like you can scale Kafka's throughput by using more partitions.

Now in some meetings some people don't like that non-answer and really want to hear some kind of performance number so I normally turn the question and ask them what their target message rates and sizes are going to be. If their answer is in the 'few thousands of messages per second' (like it is in your case if I'm not mistaken about the call in question) then, as I do know that JetStream typically comfortably provides performance well in excess of that, I can say with confidence that _at those kinds message rates_ it doesn't matter whether you use Core NATS or JetStream: JetStream is plenty fast enough. That's all I mean!

1 comments

And I would add, as soon as you are using more than one stream (e.g. do sharding using Core NATS subject transformation) because JetStream throughput scales horizontally, just like Kafka throughput scale horizontally as you add more partitions and more servers in the cluster I feel reasonably confident to say that _in most cases_ it doesn't really matter what the target number of messages per second is, as you can create a cluster large enough to provide that aggregated throughput. In properly distributed systems, the answer to the benchmark number question truly is 'how long is a piece of string'.