[hmottestad]

Does anyone know if Kafka has improved on their data loss issues since tested by Aphyr? https://aphyr.com/posts/293-jepsen-kafka

A quote from the article: "At the end of the run, Kafka typically acknowledges 98–100% of writes. However, half of those writes (all those made during the partition) are lost."

[lars_francke]

Yes, the suggestion discussed by Aphyr has been implemented. You can now set up a lower bound on the ISR size (min.insync.replicas). Together with required.acks=-1 you can wait for a message to be committed to at least min.insync.replicas nodes.

https://issues.apache.org/jira/browse/KAFKA-1555