[reilly3000]

PSA: be very careful with what you query in BigQuery on this dataset! You will be billed $5/TB scanned. I was using this dataset a couple weeks ago for fun, and ran up a $60 bill in about 10 mins. It’s important to understand that using WHERE doesn’t shrink the size of your query; instead your FROM clause should only be limited to a single date or month... not ‘*’

[jloveless]

This is EXACTLY why we moved this data into our own database for Crux.run. We took the JSON export of the Crux database and cleaned and loaded it into a database on a single server. E.g each of those queries are hitting a single box in us central

[jloveless]

There’s some of that information in the docs : https://docs.crux.run/#/introduction?id

But yeah- BQ can get very expensive very quickly if miss step

[reilly3000]

Thanks for doing so!