[baconner]

Can anybody speak to their experience using sqlite for data analysis purposes?

Am I wrong in thinking it's just plain old row oriented storage and not something more aggregate oriented?

[k2enemy]

I use sqlite to store all of my small datasets. Minimally, I think of it as a replacement for zipped CSV files. But it also has the added benefit of a relational structure and SQL.

It is super easy to access from julia, R, python, etc, so instead of importing a CSV and manipulating the data, I find it a lot easier to connect to the sqlite database and use SQL for the a lot of the joining and manipulating.

[baconner]

I see. So the benefit is mainly in having more relational structure than in actually crunching numbers. Makes sense.

[mbrzusto]

I use it quite a bit for data analysis, in particular for user-defined functions with python. Being able to explore your data (via SQL and it's powerful syntax) in addition to functions and aggregators that I define, is REALLY useful. You could do the whole thing in python (and data imports), but adding the SQL part in is so much easier than building dictionaries and filtering, sorting etc...