[mike_d]

FYI it is a little suspect to write your own testing framework and then claim a ridiculous performance gain over something like Redis. There are already widely accepted testing tools [1] you could have used.

It might make more sense to write a harness that allows your software to be used with standard Redis wire protocol so it can be properly benchmarked and compared against the dozens of existing solutions in the space.

Also it seems like you accidentally discovered the Latency Numbers Everyone Should Know [2]. Local operations are faster than network operations.

1. https://redis.io/docs/management/optimization/benchmarks/

2. https://static.googleusercontent.com/media/sre.google/en//st...

[pushkarg]

Exactly! Accessing data in RAM will be orders of magnitude faster than over the network (maybe this is a good sanity check of our benchmark numbers). The core principle behind IKV is that it allows you to access (large) data-sets without network calls.

Most DB tooling out there only works for a client-server model.

And implementing the Redis protocol would imply changing our architecture significantly and negatively affect performance (ex. a producer-consumer queue to serve requests, ser-deserialization costs).

[mike_d]

> implementing the Redis protocol would imply changing our architecture significantly and negatively affect performance

Yes, you’d be doing a fair and honest benchmark if you want to compare yourself to Redis.

[pushkarg]

I just explained how implementing the Redis protocol will inherit inefficiencies of Redis in IKV - so I don't understand how the comparison will be fair or honest.

Hope this addresses your original question about why we wrote a custom benchmarking client.

https://github.com/inlinedio/ikv-java-client/blob/master/src...

It is quite simple and is available here. There is nothing malicious in there to make IKV appear faster. Although ff you do see a bug, I am happy to fix and republish results.