[Stranger43]

And here we see this strange thing that data science people does in forgetting that 6TB is small change for any SQL server worth it's salt.

Just dump it into Oracle, postgre, mssql, or mysql and be amazed by the kind of things you can do with 30year old data analysis technology on an modern computer.

[apwell23]

you wouldn't have been a 'winner' per OP. real answer is loading it on their phones not on sqlserver or whatever.

[Stranger43]

To be honest OP is kind of making the same mistake in assuming that the only real alternatives is "new data science products" and old school scripting exists as valuable tools.

The extend people goes to to not recognize how much the people creating the SQL language and the relational database engines we now take for granted actually knew what they were doing, are a bit of an mystery to me.

The right answer to any query that can be defined in SQL is pretty much always an SQL engine even if it's just sqlite running on an laptop. But somehow people seems to keep comming up with reasons not to use SQL.