|
|
|
|
|
|
by mobileexpert
3143 days ago
|
|
|
It takes around 20 minutes to spin up HDInsight clusters. So it is not instant but I wonder if databricks on Azure can do any better given they are running in customer environments so wont have a big pool of preallocated multitenant machines available. |
|
|
They spin up the VMs with either open source Spark or the Databricks runtime - you get to choose the distribution and version before spinning up the cluster.
If you have enough workloads you can run your own pool of VMs to provide a 'serverless' experience to your Spark users: https://databricks.com/blog/2017/06/07/databricks-serverless...