Hacker News new | ask | show | jobs
by kerblang 1558 days ago
They hinted at the fact that kafka is not actually a queue, and it especially has problems with fairness if you try to use it as a queue for unequally sized "jobs" and/or unequally sized consumers. Kafka is for the exceptional case; actual message queues are for the default/general case.
1 comments

That makes sense, but it also doesn't really support the article's overall argument about the perils of "standardization without understanding". If what you're saying is true then it could very well be that the approach to standardize the queuing would have worked perfectly well had the senior dev chosen a different technology to underlay his library (ie not Kafka), right?
Yes technically the Kafka matter is a secondary to the author's main point, which is understanding each problem first. So yeah I would not waltz in and say, "Oh, no need for details, obviously you need to standardize on RabbitMQ not Kafka," without a second thought, even though it's likely that's roughly where we're gonna end up.

But even then, I am not going to write a custom library when the drivers for either product are perfectly adequate. Standardization works best when standards are flexible, and trying to force my homemade library on every team is inflexible (usually it's just arrogance).