Hacker News new | ask | show | jobs
by felipesabino 3866 days ago
I've been using kafka 0.8.2 for some time now together with Node.js for both consumer and producer.

Although the producer side is quite simple to use and have more than one option available, the consumer side there is only one project that is "maintained" and works [1][2], all other opstions either only have producer available [3] or have not received a commit in years [4].

I am a bit disappointed about how little attention Node.js with kafka had so far as there are a lot of issues on keeping connection alive and rebalancing that made it really hard to trust the system and automate zero downtime deploys.

Although I still hope all these changes in 0.9 new consumer API solve these issues, I am really happy about the decision to be backwards compatible, making the transition/upgrade a much more smooth process

> To ensure a smooth upgrade paths for our users, the 0.8 producer and consumer clients will continue to work on an 0.9 Kafka cluster.

[1] https://cwiki.apache.org/confluence/display/KAFKA/Clients#Cl...

[2] https://github.com/SOHU-Co/kafka-node/

[3] https://github.com/sutoiku/node-kafka

[4] https://github.com/wurstmeister/node-kafka-0.8-plus

1 comments

Your critique is well received. The Apache Kafka project has support for the Java clients and the non-java clients will be developed and available in a federated manner. At Confluent, we are focused on providing first class non-java clients that are API and functionality compatible with the java clients. Forthcoming releases of the Confluent Platform will include a C/C++, python and node.js client. Stay tuned http://www.confluent.io/developer#download
For those that don't realize it, Neha and Jay were two of the main developers who wrote Kafka.

Thanks for the heads up!

We've been using Kafka for almost two years already. Still lots of our codebase is Ruby, and Poseidon is not really that great client library. It's slow and it's not threadsafe.

In our Scala-side we're happy with the current offering.

The C/C++ client, will that be librdkafka? If I need a high-quality client for another language, would you recommend building that using librdkafka or the rest proxy?
Is anyone working on Go bindings for 0.9?