Hacker News new | ask | show | jobs
by numlocked 4232 days ago
Should I think of this essentially as an abstraction that can replace my asynchronous task queue + workers (e.g. RabbitMQ + Celery workers, obviously assuming you aren't using MQ for additional messaging)? I hate managing those pieces and would be happy to hand that infrastructure to Lambda, but are there additional benefits or use cases that are opened up?

I guess I would have expected others to describe this the same way ("replaces your distributed task queue"), but since I'm not seeing that description I wonder if I've misunderstood.
2 comments
jberryman 4232 days ago
I guess maybe part of what you might be missing is that these "Lambda functions" can (will be able to?) be kicked off by all sorts of different AWS events, so it can replace queue + workers where you're generating tasks from your own code, but it might also be able to replace the code that generates those tasks too.
link
numlocked 4232 days ago
Now that you say it, I'm sure we'll see wrappers to support this type of thing. For instance, I could imagine a django library that provides a declarative way to wire Django signals to AWS Lambda functions.

lambda_signal(Model, signal_name, lambda_fn_name)

When that signal fired on the model, the model would get serialized to JSON and sent to the lambda function. It'd be fun to whip this up if no one beats me to the punch.

lambda_signal(User, 'post_save', 'sync_to_mailchimp')

link
crucialfelix 4232 days ago
Rabbit and celery is the backend system I have now. I was doing a lot of things on demand, event driven, but this resulted in overloading the database ( currently my system constraint ) or in elongating the queue too much. Some event would flood the queue with thousands of tasks. Now I have a periodic task that on each iteration chooses the most pressing task. And the system is stable but I have a bad backlog.

So I'd love to have a reactive task system where I didn't have to worry about the resource usage. Think how much precious developer time I've wasted just managing resource expenditures. Much of my coding this week has been just trying to optimize for this stupid constraint. How many processors do I have, how much disk io, is the memory getting close to swapping ?

I for one welcome our new cloud overlords.

link