|
|
|
|
|
|
by darth_avocado
844 days ago
|
|
|
BQ charges you based on the volume of data being scanned. I think this is a situation which involves scanning the whole dataset again and again without fully understanding how it works. I’ve worked with much larger datasets on BQ (petabyte scale) and managed to not spend more than $1000 in an hour. Also, BQ tells you how much data will be processed BEFORE you run the query, which makes it easier to understand the cost implications. Again, you could fit the whole dataset in memory in an EC2 instance and do your thing. |
|
|