|
|
|
|
|
|
by apurvamehta
3281 days ago
|
|
|
Points 1 & 2. There is no direct communication between a producer and consumer. The producer writes to the broker, the consumer reads from the broker. There is a detailed flow diagram for the producer side operations in the article, and this deck has more of the details of how both the producer and consumer work: https://www.slideshare.net/apurva2/introducing-exactly-once-... 3. Yes, it has been tested empirically. Quoting from the article: > We wrote over 15,000 LOC of tests, including distributed
> tests running with real failures under load and ran them
> every night for several weeks looking for problems. This
> uncovered all manner of issues, ranging from basic
> coding mistakes to esoteric NTP synchronization issues
> in our test harness. A subset of these were distributed
> chaos tests, where we bring up a full Kafka cluster with
> multiple transactional clients, produce message
> transactionally, read these messages concurrently, and
> hard kill clients and servers during the process to
> ensure that data is neither lost nor duplicated. |
|
|