[1e10]

1tb is nothing these days. If you insist on cloud the hetzner could be best bang for buck. Otherwise a similar desktop system can be acquired for less than 1000 usd.

I’d start with solr or elasticsearch and a simple indexing script (home rolled python script).

Then you can use solr admin or something like Jupyter for iterative querying.

I’m not an expert on index tuning, but you might even be able to dump it all into postgres with json types.

Best of luck!

[johnnycarcin]

Yeah, the amount of data is pretty small in the grand scheme of things, maybe that is why i'm getting so hung up haha. Elasticsearch was actually the first thing I thought of so maybe I'll just go with that and see what happens...