Hacker News new | ask | show | jobs
by michael_j_ward 1490 days ago
If one does not need petabytes of scale, but am otherwise interested in the data lineage / observability / workflow being sold by databricks, what would you suggest?

Some evaluation Criteria:

- Ease of maintenance and operation is almost paramount.

- It's fine if the solution never lives anywhere but 1 single virtual server that scales vertically (data might grow to a couple TB, but not PETA BYTES)

- Similarly, 20 9's is not a criteria. If the machine fails and it takes an hour till someone goes and re-deploys, that's fine.

- Declarative, reproducible deployment with an easy upgrade story would be great

- Ideal if the deployment can be run locally for quick developmnet

1 comments