|
|
|
|
|
by slap_shot
3250 days ago
|
|
I'm a co-founder of a stealth-stage company that helps data analysts/data engineers build data pipelines. Every "task" that can be done in our framework is essentially just an image that can be reused over and over with different settings. We deploy these tasks across Kubernetes clusters on AWS, GCP, and Azure. Since these tasks are schedule irregularly and are short lived, we had to do a lot of work to dynamically scale the nodes up a head of their demand and down after, and we typically have to pay for at least 10 minutes of usage no matter how quickly the job finishes. This "pay-by-the-second" will be a huge win for us. Most of our tasks deal with S3/Redshift or GCS/BigQuery, do we can't immediately use this. But as we onboard more clients working with Azure Storage/Data Lake/Data Warehouse I see some big operational gains for us. Here's hoping we see similar developments across the other major cloud providers. Very impressed with Azure's development in the last 3 years! |
|
On the downsides, they're small and they have one data center, and they're not Microsoft. But their tech is open source.