[aphyr]

If you're wondering about the workload, this is the trivial Riemann config this benchmark uses:

  (streams
    (rate 5 (comp prn float :metric))

Which means for each five-second interval, sum all the metrics of the events flowing through this stream, divide by the time elapsed, and print that rate to the console.

I'm using this setup to put the heaviest load possible on Riemann's client and TCP server while I optimize those layers--it's not meant to stress the internal stream library, the state index, or pubsub. When I start optimizing those components, I'll have more "real-world" numbers to report.

I should also explain that this particular post explores the high-throughput, high-latency range of the client spectrum. End-to-end TCP latencies (not counting wire time) for single-event messages are on the order of ~100 microseconds-1ms, with occasional spikes to ~30ms depending on JVM GC behavior.

[spullara]

You are likely doing something wrong. My in-memory redis server clone that also uses netty reaches 1.5M requests/s on an arguably heavier workload with a more complicated protocol. I'd love to help you optimize it if it matters.

[aphyr]

Thanks for the offer! I, uh, certainly didn't mean to claim that Riemann is extraordinarily fast. Up until this point I've been working as hard as I can just to make the darn thing turn on and accomplish something useful. This is quite early in the optimization process, haha. :)

The principle bottleneck in this particular test is actually the client--Riemann itself spends ~94% of its time waiting on epoll in this test. The client is a total hack using Java OIO, calling flush() on every message, Nagle's algorithm disabled for low per-msg latencies... it's a wreck, haha. Part of next week's optimization push is replacing it with a Netty client and tuning TCP options for various workloads.

[no7hing]

Things will probably look better once you switch your client implementation to Netty as a similar test I did with Netty 4, Protobufs and batching of messages produced low seven figure numbers too.

[spullara]

Ah ok, my test was run using the C redis-benchmark client.

[thebluesky]

Is the source available anywhere?

1.5M locally or over a network? What is the size of each message?

[spullara]

http://github.com/spullara/redis-protocol — server module

The test was run with "SET key value" which in the redis protocol is something like 15-20 bytes / message.