[fizx]

Seems like a useful reworking of SQS, but all the hard work is being done in the client: "client library automatically handle complex issues like adapting to changes in stream volume, load-balancing streaming data, coordinating distributed services, and processing data with fault-tolerance."

Unfortunately, there's no explanation of the mechanics of coordination and fault tolerance, so the hard part appears to be vaporware.

[vosper]

> Unfortunately, there's no explanation of the mechanics of coordination and fault tolerance, so the hard part appears to be vaporware.

I think it's unfair to call it vaporware - Amazon doesn't tend to release vaporware. You can also be fairly confident this has been in private beta for some time, so we'll probably see a few blog posts about it from some of their privileged (big spending) clients - typically someone like Netflix or AirBnB. But I agree it would be nice to get some more information on the details.

As for the client library handling load-balancing, fault tolerance, etc - that might not be ideal, but as long as I don't have to do it myself then it might be okay.

[fizx]

The client handling it is ideal from a systems perspective, because the app won't forget to be fault tolerant on its connection to the server.

Its less ideal from a maintenance perspective, because there will need to be feature-rich clients in Java and C (with dynamic language bindings). Applications will be running many many versions of the clients. Also, for coordination, the clients will need to communicate, so there may be configuration and/or firewall issues for the app to resolve.

It will be interesting to see Amazon make this tradeoff for what I believe is the first time.

[aluskuiuc]

It's not exactly the first time, but close - the Simple Workflow Service has client helper libraries for both Java and Ruby.

[javajammer]

The currently available docs reveal the client-nodes coordinate through a DynamoDB table. Processing with the library yields "at least once" semantics.

http://docs.aws.amazon.com/kinesis/latest/dev/kinesis-record...

[fizx]

thanks!