|
|
|
|
|
|
by jiangplus
2235 days ago
|
|
|
Looking at the TileDB docs, I like the idea that you can store and manipulate array efficiently. It works like a columnar storage, but organized in tiles. I am not clear how it could scale to multiple nodes and how to do queries. Maybe it should be integrated into postgres. |
|
|
TileDB fully supports multi-writer/multi-reader, so scaling compute to multiple nodes is mostly about choosing the computation layer. We have integrations with Spark, Dask and PrestoDB for distributed compute. In our cloud product we also offer serverless UDFs and SQL (via MariaDB) to allow for scaling out computations elastically without managing your own clusters.
On the postgres side, we are looking at eventually adding a postgres storage engine, now that the new storage support is in PostgresSQL 11. To start with we have built a storage engine for MariaDB, as MariaDB has an excellent storage engine API, which I have past experience with. We hope to upstream the storage engine to MariaDB after the MariaDB 10.5 release.
MariaDB integration: https://docs.tiledb.com/mariadb/
Spark integration: https://docs.tiledb.com/spark/
Dask integration: https://docs.tiledb.com/dask/
PrestoDB integration: https://docs.tiledb.com/prestodb/
Cloud Docs: https://docs.tiledb.com/cloud/