Hacker News new | ask | show | jobs
by dailypeeker 2061 days ago
Is this Amazon's response for Kafka?
5 comments

There's some overlap in use case, but Amazon already has a more compatible managed Kafka service called Amazon MSK.
I would have assumed Google's pub/sub, which released a similar feature a few weeks ago.

https://news.ycombinator.com/item?id=24658132

That, or they both had convergent evolution and came to the same set of requirements, maybe from some big user at the same moment contacting both of them a year ago, which actually feels really likely to me (more likely than them responding to anything).

"You can process up to 300 transactions per second (TPS) per FIFO topic or FIFO queue"

I want to say Kafka can do significantly higher messages per second on a single partition (and significantly higher than that on a many-partition topic). I don't think it can compete for a lot of use cases as a result. If I were in the market for managed Kafka, I think I would just go straight to Confluent.

> providing strict message ordering and deduplicated message delivery to one or more subscribers.

this is hard yo

I still have nightmares about rendezvous pubsub.
Kafka and SNS only conceptually intersect, SNS is more for intra-service and endpoint notification at huge scale (millions of subscribers) and is directly integrated into a variety of AWS services (e.g. S3 event notifications). You can set up a topic, subscribe a customer via SMS and send a message in just a few lines of python:

    $ python3
    >>> import boto3
    >>> sns=boto3.client('sns')
    >>> sns.create_topic(Name='DopeService')
    >>> sns.subscribe(TopicArn='arn:aws:sns:us-east-1:xxxxxxxxxxxx:DopeService',Protocol='sms',Endpoint='+18008675309')
    >>> ...<repeat for other SMS #'s>...
    >>> sns.publish(TopicArn='arn:aws:sns:us-east-1:xxxxxxxxxxxx:DopeService',Message='sup')
If you're a masochist you can have Amazon notify everyone on the topic every time someone uploads to your bucket

    >>> sns.set_topic_attributes(TopicArn='arn:aws:sns:us-east-1:xxxxxxxxxxxx:DopeService',AttributeName='Policy',AttributeValue='<policy>')
    >>> s3.put_bucket_notification_configuration(Bucket='dopebucket',NotificationConfiguration={<notification config>})
You send directly to HTTP/HTTPS service endpoints, other SNS topics, Lambdas, SQS queues and mobile app push notifications. SNS is more about signaling than a raw message pipe. Relative to Kafka it's basically apples and oranges.
SNS is intended for different use cases. Kinesis and other AWS products provide services in line with Kafka.
If you don't count Kinesis and their literal Kafka offering, MSK.