[necro]

Last time I used Redis I was surprised to determine to my surprise that Redis was single threaded. Of course I could have just RTFM but I assumed incorrectly.

This means that if you have part of your application that requires fast consistent GETs, and then another application does a slower SORT, UNION, DIFF, etc, on the same db or even other dbs on the same Redis server, EVERY other client request has to wait for this slower command to finish. http://redis.io/topics/latency

This is something that one really has to engineer around in order to use it in an environment that requires performance and consistent latency. In our case of 1000s req/s it was just unacceptable to have the latency be affected, sometimes by 10 times, by a slower command.

I do love all the sort, diff, union commands.

[jbert]

If the two datasets with different access speed requirements are disjoint, you can just run two instances of redis. One for the high-latency gruntwork, one for the low-latency GETs.

If the datasets aren't disjoint, then you're trying to do fast and slow ops with the same data, which - if you need accurate values - is going to be mildly hairy even if multithreaded, since you'll need to somehow lock the data while you do the slow op (which will exclude the GETs, causing high latency), or you'll need some kind of transaction-based stable view to operate on (e.g. transactional memory?)

[necro]

Very rare data access is disjoint, unless you're only doing key/value put/get. I think the interest of Redis is that it has many other features than simply put/get, and all those sorts, diff, etc typically would work a set of data that is being written in.

For sure having multiple instances will help some of this, but adds more complexity. Do you have your app write to multiple instances, and then read low latency from one, and read high latency from another? Is that data now consistent? Do you setup Redis replication and make sure that works right and then read from different replicas? Or perhaps you engineer some queue that does not block writes, groups them together and writes to Redis in a separate thread. Then you have to maintain all this and make sure it's correct, back it up, what are the corner cases, failure modes, etc.

From my experience, if you want to engineer things well, you end up essentially building out the same sub systems that a larger db engine has. Say Innodb. I'm smart enough to know that I'm not smart enough to build a one off complex system more correctly than really smart people that have been iterating over many years and improving things on something like innodb.

There are very rare, very specific cases where I would use redis over something else if I was building something realtime, large and important.

[opendais]

That may be your experience but...

I suggest you google YouPorn's architecture.

I think its a domain/scale issue. It isn't a 'everything must become a more complex db engine to be engineered well' issue.