[thinkersilver]

Great article. I think you might be taking a subtle point in the article for granted. Namely, the ability to assemble a fairly complex distributed system from open source components.

Democratisation of complex machinery like the raft and consensus algorithms, schedulers, append logs, query optimisers and so on is a superb thing. It is something that has only been possible in the last 3/4 years and would have been devilishly difficult before then without significant upfront work.

Fetishising complexity might be a bit strong here. Seeing more posts like this is a good thing.

[mLuby]

You make a good counterpoint: it's awesome that this can be done concisely enough these days to fit in a hobby blog post. That's new and demonstrates the approachability of these tools.

At the same time, OP is right: a business doing this is making poor engineering decisions.

[kjeetgill]

We don't know that yet! The post explains what they did but not the why of the problem. Maybe they have a good reason, maybe they don't.

But no need to jump to conclusions. I've seen plenty of bs fragile ducktape databases. Some of them exist because because someone wanted to hand-roll everything. And some exist because they were afraid to invent anything and ended up hacking together a bunch of garbage MySQL nodes. Because they didn't want "reinvent the wheel".

Specifics matter!

[humbleMouse]

I agree my comment may be a knee-jerk reaction, and it is quite a good technical article with clear examples.

I also agree that the open source tech used in this solution is very powerful and it is great we are at a point where a solution like this could exist. I love avro schema registries and kafka just as much as the next guy.

[kjeetgill]

So question: is there a good reason to use Avro over protobufs?

Just from a hand wavy aesthetic perspective I like protobufs but I'd like to get a clear picture.