[mu53]

I think it'd be very easy to write your own. I used postgres subscribe/listen built in combined with a database table to get a distributed message system.

Writing a distributed, scalable system is really hard, and beyond the API, that is the real value for kafka

[local_crmdgeon]

>I used postgres subscribe/listen built in combined with a database table to get a distributed message system.

Every single person I know who's done this says it was a fantastic decision, and the "eventually I'll have to migrate to X" never came.

[iudqnolq]

how do deal with connection limits? you can't listen through a pooled connection.

[chrsig]

It's relatively easy -- removing any networking requirements drastically simplifies the problem. There's still some non-trivial bits that vary depending on granularity for concurrency.

It's a weekend project to demonstrate the concept, maybe a few weeks to really flesh it out and iron out quirks. I imagine if you're willing to use sqlite as a backend for persistence, it gets a bit easier.

[psd1]

You may be right. However, consider the directorial perspective.

You have employees - you try to get and retain the best talent you can. However, every human has strengths and weaknesses, and these may not all be fully visible to you.

Rolling your own vs buying off the shelf is a gamble on future outages.

Will a third-party support and fix the issue, or have a strong community that can help you work through the issue?

If your best engineer builds something that works for long enough to become entrenched, but then carks it, will your best engineering talent be able to resolve the issue? If your rockstar quits, does the team have to pick through the halls of Cthulhu? Does your organisational ignorance of kernel networking suddenly become painfully apparent?

Remember, you need to be twice as clever to debug your code than to write it...