[pearjuice]

I don't know how well the results hold up after a year, but according to Redis the Dragonfly performance test is biased and with Redis configured properly it reached higher throughput than Dragonfly. YMMV but just putting this up here. Personally I never used Dragonfly so I wonder if the "marketing metrics" actually hold up in production.

https://redis.com/blog/redis-architecture-13-years-later/

[gadelat]

Redis Cluster has lot of limitations though. It's unusable for multi-key operations, no scan, no transactions, single database only, client has to support it, the way it works makes it unusable when connecting to it outside the network (see https://redis.io/docs/management/scaling/) etc. At that point RedisCluster is not Redis anymore and it's disingenuous to call it that. I would rather have slightly lower performance and not have to deal with those limitations AND not have tot deal with orchestration.

[youngtaff]

Yeh but isn’t the Redis one just a biased?

What might have been interesting would be to test on a range of cores / clusters, and consider the overhead of managing 1VM vs 64VMs etc.

[arp242]

The Dragonfly benchmark runs one Redis instance on a 64-CPU machine and compares it with one Dragonfly instance on the same machine.

But there is nothing stopping you from running 64 Redis instances on one machine if it has 64 cores, which is what Redis did (actually, they ran just 40). That actually seems like a nicer design overall, as it scales "naturally" to multiple machines without any extra effort/code, it keeps the code simpler, you can also have one of these Redis instances segfault without bringing your entire cache down.

Other than that, they seem to have run the same benchmark. YMMV for other types of workloads of course, and perhaps Dragonfly could be configured better in some way.

Either way: it seems the Dragonfly benchmark is not just biased, but highly misleading. And while the Redis benchmark may be biased, it certainly doesn't seem highly misleading.

[PlutoIsAPlanet]

To me, spinning up multiple copies of the database is cheating. You're comparing a box of Apples to a single apple.

Yes, using a Redis cluster is the only way to get Redis to actually use system resources effectively, but its a relatively complex thing to create and manage compared to just running 1 server.

[arp242]

I don't think it's cheating at all; it's how its designed to work.

If you want to say "but this is more difficult": okay, fair enough (although in my experience it's not difficult at all), but then say that instead of posting a misleading benchmark which runs Redis in a way it's not supposed to run. You can place all sorts of artificial "yeah but I don't want to do it like this" constraints on all sort of things.

[romange]

hmm and ships are designed to sail, yet you use planes to cross atlantic. Nokia was designed as strongest and most affordable phone, yet you use Iphone that costs 1000$. it's not about how it was designed but whether it addresses your current needs. Developers do not want to manage a cluster of single cpu processes. Not on their laptops and not in the production. And it's not just about management complexity. See this https://github.com/dragonflydb/dragonfly/issues/1229 and it's just one example. Single cpu - is just not enough for today use-cases.

[arp242]

That may all very well be the case – let us assume it is for the sake of the argument although I have some comments about that as well – but that still means the argument is "Redis is too complex to run on multiple CPUs" and/or "Redis is poor for these workloads" (I didn't investigate that issue in-depth), and not "Redis is unable to do much work with this very powerful AWS instance". There two are very different things. There is no nuance anywhere in the benchmark. A reader might very well believe that this is all the performance they're going to get out of Redis on that machine, which that's clearly not the case.

> Nokia was designed as strongest and most affordable phone, yet you use Iphone that costs 1000$

Actually I have a Nokia :-)

[pritambaral]

> To me, spinning up multiple copies of the database is cheating.

What if the database was designed to be run that way?

> You're comparing a box of Apples to a single apple.

Precisely. Dragonfly is a box of apples. Redis is a single apple that can be put in a box with other apples. If you run a "benchmark" comparing your box of apples against a sole apple, you're being either stupid or dishonest.

[Temporary_31337]

At least on AWS it is kind of hard to get 40 tiny VMs with sufficient speed on the infra side. Given that laptops get 40+ vCores these days, I think a single instances od anything should have some multi threading.

[detaro]

The comment you replied to explicitly said (so you don't even have to read the redis article, which also clearly says so)

> The Dragonfly benchmark runs one Redis instance on a 64-CPU machine and compares it with one Dragonfly instance on the same machine.

They were not running 40 tiny VMs!

[ec109685]

The should have chosen a 1024 core box and really shocked the world.

[detaro]

> and consider the overhead of managing 1VM vs 64VMs

They clearly are not running 64 VMs in the test they are describing.

They compare both databases on one VM of the exact same size, both deployed as their makers recommend to deploy them.