This looks great, I’m just wondering what Amazon’s motivation for open sourcing it is. It seems like some pretty critical secret sauce for making services like Lambda and Fargate both secure and efficient.
Google recently open-sourced Gvisor, which although implemented differently solves a similar problem. Possibly Amazon wants to encourage other vendors to build integrations with Firecracker rather than Gvisor.
Right. In the end, AWS saw containerization as an existential threat, and serverless is its response to the commoditization of AWS (and other specialized cloud vendors) by containerization technology. Serverless helps AWS re-couple your application back into the specialized AWS vendor environment, once more requiring you to keep specialized and costly AWS-specific knowledge on-hand in order to build and deploy your application. It gives them plenty of room to advertise and provide all of those edge case services to you once more (and their usage charges) and also helps them prevent you from treating AWS like a rack of networked CPUs to serve as the substrate for your application containers.
People (mostly AWS folks -- dig a little deeper into who is writing much of the serverless blog posts out there) keep pushing the "serverless is containers" but that's just a tactical response. Add a layer of abstraction and it's very clear why AWS is betting so hard on serverless. Originally, AWS commoditized the old datacenters by providing the same network/CPU substrate, but at a higher cost because you outsourced the management of those resources to AWS. And AWS slowly dripped out new and convenient services for your application to consume, allowing you to outsource even more of your application needs to this one vendor. And while the services offered by AWS were just a little bit different, they were functionally similar. And that's how you locked yourself into using AWS instead of CoreColoNETBiz or whatever datacenter you were using before. I remember one of the first major outages of us-east-1, which caught most of the internet with its pants down (interestingly, the answer was just to give more money to AWS for multi-region redundancy). AWS had a pretty good thing going: Outsourcing the management of all those resources to AWS is expensive! But that's when containers came along and people at AWS started to take notice. With containers, people could de-couple their applications from Dynamo and Elastic Beanstalk and VPC and all those specialized services that cost so much time/money. Instead, you could just cram all that shit into containers, without needing to set up IAM roles or pore over Dynamo documentation or dump so much time into getting VPC set up just right. And that's the whole point of containerization: Easily build your services in a homogeneous environment with exactly the software you want to use and eliminate that technical debt of vendor lock-in and the enormous cost center of specialized vendor knowledge (e.g. Dynamo, IAM, VPC, etc etc). Treat the cloud -- any cloud -- like a bunch of agnostic resources. Docker commoditized the commoditizers.
And serverless is how AWS plans to get you to re-couple your application tightly to their specialized web of knowledge and services. They get to say that you're still using containers, but they need to gloss over the fact that you're locked into the AWS version of containers. You cannot "export" your specialized AWS-only knowledge of Fargate or Lambda or API Gateway or ECS to Google Cloud or Azure or some dirt cheap OVH bare metal. You're tightly re-coupled to AWS, having bought into their "de-commoditization" strategy. Which I need to stress is totally fine, if you're okay with that. It just needs to be made clear what you are trading off.
Having worked at AWS, I have to disagree with you.
No-one that I worked with saw containerization as a threat. And why would they? At the VM level you can already paper over differences between cloud providers and I don't think that anyone at any of the large cloud providers lies awake at night worried about this.
I also don't understand why serverless would couple you to a particular cloud provider. All the big cloud providers provide serverless features and it never takes long to see feature parity.
What ties you to a cloud provider (or any company) is when you use features unique to that provider. And presumably you're using those features because the value they add outweighs the perceived costs of lock-in or the cost of implementing it yourself.
The point is that each serverless implementation is different enough that even if you are using the same feature, the cruft around that is different enough to provide an certain amount of lock in.
Sure and this is to be expected. It costs time and money to align yourself with someone else's implementation and unless your customers demand alignment (e.g. S3-compatible storage interfaces), you're probably not going bother.
Again, this comes down to cost-benefit calculations. If some companies find that proprietary feature X from cloud Y provides a bigger (perceived) return on investment than not using feature X, then they are likely to use it. If company X later shafts them, they have to swallow more costs to migrate away but hopefully (for them) they took this possibility into consideration when they made their original decision.
S3 is the bigget coupling, or data storage. Why do you think containerization threatens AWS? It doesn't. Vendor lockin happens naturally, IAM are a vendor lock in feature, and I didn't see a lot people even mention about it. Migration off those sruff is incredibly hard and things can easily go wrong.
https://cloud.google.com/blog/products/gcp/open-sourcing-gvi...