[doyoubi]

Redis 6 has already supported multiple threads for IO. The memory storage engine is still inside a single thread but I think it's fine.

[throwaway81523]

There is a parallel but less fancy reimplementation of Redis called Pedis:

https://github.com/fastio/1store

It's built using Seastar, which is sort of an ugh but cool unholy marriage of node.js and C++14 with an event loop per thread, and message passing between threads. It predates C++20 coroutines I guess, so you pass callbacks around like in Node using C++ lambdas. Maybe they have updated that by now.

They have also done a version of memcached using DPDK. I don't see a Pedis one, but check out their memcached one (scroll down): http://seastar.io/

[derefr]

> It's built using Seastar, which is sort of an ugh but cool unholy marriage of node.js and C++14 with an event loop per thread, and message passing between threads. It predates C++20 coroutines I guess, so you pass callbacks around like in Node using C++ lambdas. Maybe they have updated that by now.

That seems like a lot of work just to badly recreate Erlang. (Which, yes, doesn’t let you define your own native data structures in-and-of-itself — but there’s always NIFs, e.g. https://blog.discord.com/using-rust-to-scale-elixir-for-11-m....)

[EdwardDiego]

Wow that's a product name horribly prone to typo/auto-correct awkwardness.

[rubyist5eva]

At first I was like, “heh, parallel redis so pedis”…then I said it out loud and I was like….oh my

[doyoubi]

Cool! Thanks for sharing.

[lrem]

Do you mean that a single CPU can saturate the throughput of a machine's RAM? That's a bit surprising, given that current crop of Intel CPUs does non uniform memory access. Or, is that bandwidth so much above any I/O that it doesn't matter?

[doyoubi]

Nop. It's that a better solution would still be using multiple Redis instances on each physical server. Trying to support multi-threads in the storage engine requires complex synchronization implementation. Sharing nothing between each small instance achieves higher performance and is a lot simpler. Also, FULL SYNC between masters and replicas can be uncontrollable. And smaller memory size for each instance alleviates the performance impact from FULL SYNC.

[ohthehugemanate]

not to mention, multiple instances follows the indistry direction in horizontal scalability. Common advice when building for k8s style cloud environments is to target single core, n instances, to maximize scaling flexibility and atomicity.