[gizdan]

I don't know a whole lot about Redis, so I'm struggling to understand what this is. Am I correct in saying that this is just the clustering mechanism? So I'd need to run Redis instances, and use this to cluster them? If so, would this work with KeyDB (would that even make sense)?

[doyoubi]

Yes, that's accurate. This project runs on top of Redis as Redis has already done a good job as a memory storage engine. And https://github.com/doyoubi/undermoon-operator will help run it with Redis instances together, so you won't need to set them up manually. And yes, it can work with KeyDB in theory.

[lrem]

Is Redis still single-threaded? I thought that would become a bit of a problem by now.

[doyoubi]

Redis 6 has already supported multiple threads for IO. The memory storage engine is still inside a single thread but I think it's fine.

[throwaway81523]

There is a parallel but less fancy reimplementation of Redis called Pedis:

https://github.com/fastio/1store

It's built using Seastar, which is sort of an ugh but cool unholy marriage of node.js and C++14 with an event loop per thread, and message passing between threads. It predates C++20 coroutines I guess, so you pass callbacks around like in Node using C++ lambdas. Maybe they have updated that by now.

They have also done a version of memcached using DPDK. I don't see a Pedis one, but check out their memcached one (scroll down): http://seastar.io/

[derefr]

> It's built using Seastar, which is sort of an ugh but cool unholy marriage of node.js and C++14 with an event loop per thread, and message passing between threads. It predates C++20 coroutines I guess, so you pass callbacks around like in Node using C++ lambdas. Maybe they have updated that by now.

That seems like a lot of work just to badly recreate Erlang. (Which, yes, doesn’t let you define your own native data structures in-and-of-itself — but there’s always NIFs, e.g. https://blog.discord.com/using-rust-to-scale-elixir-for-11-m....)

[EdwardDiego]

Wow that's a product name horribly prone to typo/auto-correct awkwardness.

[rubyist5eva]

At first I was like, “heh, parallel redis so pedis”…then I said it out loud and I was like….oh my

[doyoubi]

Cool! Thanks for sharing.

[lrem]

Do you mean that a single CPU can saturate the throughput of a machine's RAM? That's a bit surprising, given that current crop of Intel CPUs does non uniform memory access. Or, is that bandwidth so much above any I/O that it doesn't matter?

[doyoubi]

Nop. It's that a better solution would still be using multiple Redis instances on each physical server. Trying to support multi-threads in the storage engine requires complex synchronization implementation. Sharing nothing between each small instance achieves higher performance and is a lot simpler. Also, FULL SYNC between masters and replicas can be uncontrollable. And smaller memory size for each instance alleviates the performance impact from FULL SYNC.

[ohthehugemanate]

not to mention, multiple instances follows the indistry direction in horizontal scalability. Common advice when building for k8s style cloud environments is to target single core, n instances, to maximize scaling flexibility and atomicity.