Hacker News new | ask | show | jobs
by alexatkeplar 3101 days ago
With Kinesis, you would just use the Kinesis Client Library (https://github.com/awslabs/amazon-kinesis-client-python) which would automatically handle committing the offsets to DynamoDB.

Home-rolling a checkpoint-free event pipeline is a rookie mistake; it's a pity they didn't come across our Snowplow project (Apache 2.0 event pipeline running on Kinesis, Kafka and NSQ, https://github.com/snowplow/snowplow/).