Hacker News new | ask | show | jobs
by datadrivenangel 880 days ago
How do you keep the pool costs manageable?

I see a lot of companies that get sold on Databricks and then are surprised by the cost.

1 comments

Pool costs become more manageable as you have more clusters sharing the same pool. You can also have it have no incremental cost increase by setting the timeout to be 0, but it makes it less useful. You can have more clusters take advantage of it by using the same instance families. If you purchase reserved instances from Azure/AWS, you might as well make a pool with those as well. You may also want to check out fleet instance types.