[threeseed]

No.

EAVT is great as an intermediate format but it is absolutely useless to query for since most of the time you are trying to find a set of attributes for a given entity i.e. full table scan.

What you want is a "wide table". One entity column and all the attribute columns to the right. Often with most of the values set to null.

This is the dream use case for MongoDB since it you can ignore sparse values yet when you query it via their drivers it will appear as a wide table. You can't do this at all in PostgreSQL since you will hit a column limit.

[flukus]

> EAVT is great as an intermediate format but it is absolutely useless to query for since most of the time you are trying to find a set of attributes for a given entity i.e. full table scan.

This is what indexes are for. An index on the entity id should avoid any full table scans.

[threeseed]

EAVT table with 100 million entities and 10000 attributes = one trillion row table.

And you want to build indexes on half the table ?

Good luck with that.

[flukus]

> Often with most of the values set to null.

Your math is at odds with your own requirements, null values don't need a row.

[threeseed]

You clearly don't understand what you're talking about.

Sparcity is an issue for the wide table not the EAVT form.

[kuschku]

> You can't do this at all in PostgreSQL since you will hit a column limit.

JSONB is designed for exactly this, isn’t it?

[threeseed]

Sure. But MongoDB is far better at scaling, has infinitely better drivers (including Spark) and is about an order of magnitude faster than PostgreSQL for partial updates.

The lack of a Spark driver alone renders PostgreSQL useless for most companies.