Hacker News new | ask | show | jobs
by morelisp 1610 days ago
Independently of application logic (which also sometimes uses it), it's the primary mechanism in Kafka for handling high-throughput consumers while still recovering from failure. Consumers grab a batch of e.g. 1000 events, checkpoint every X seconds while processing them; if they die the events are still there and they restart from the last checkpoint.

It also means every message in Kafka is "addressable" via topic/partition/offset which lets you refer to "foreign" messages etc.