Hacker News new | ask | show | jobs
by alexatkeplar 4110 days ago
Hey 0x74696d - Snowplow co-founder here. Very cool post! The Snowplow Kinesis architecture gives you "fan-out" for free - you could write your bookmarking service as a Kinesis KCL app which reads from the Kinesis enriched event stream written by our Kinesis Enrich (https://github.com/snowplow/snowplow/tree/master/3-enrich/sc...). In this case you'd be using our Scala Stream Collector (Spray with a Kinesis back-end), not the CloudFront CDN Collector.
1 comments
0x74696d 4110 days ago
Nice. If I was starting this project today (it's pretty mature at this point) that'd definitely be something I'd look into.

That being said, when Kinesis previewed it had only Java bindings for KCL. Not sure if that's still the case, but that'd be a limiting factor for our shop unfortunately.

link
alexatkeplar 4110 days ago
You're right 0x74696d - originally the KCL was Java only. The Java KCL now includes something called the MultiLangDaemon, which means you can write apps in other languages. There is an official Python KCL (https://github.com/awslabs/amazon-kinesis-client-python) but no others I know of yet. There's also AWS Lambda for processing Kinesis streams with JavaScript, and of course you can use Storm or Spark Streaming, although those are JVMish too.
link