Hacker News new | ask | show | jobs
by dpkp 3658 days ago
No apology required. Though note that pykafka requires >=0.8.2 , and is only forwards compatible w/ newer brokers. This means that pykafka implements the 0.8.2 feature set. Newer brokers support that feature set, but you are not taking advantage of 0.9 or 0.10 features if you connect to them. kafka-python, on the other hand is both forwards and backwards compatible. It supports all feature sets: from no offsets in 0.8, to zk offests in 0.8.1, to kafka offsets in 0.8.2, to group management in 0.9, to message timestamps and relative-offset compressed messages in 0.10. The feature set to use is chosen based on the broker version we're connected to. As far as I know, no other client supports this approach -- not python, not java, etc. [Though KIP-35 should open this up to other clients for backwards compatibility starting at 0.10]
1 comments

Pykafka does currently have support for 0.9 group management, and we intend to add support for message timestamps and the other new 0.10 features. We're not, however, detecting the broker version and turning on features on that basis. Instead we prefer to let the user explicitly enable the features they're interested in using.