Ah yes, I see now. Yeah makes no sense to offer a REST response for each request.
On that note, what would the processing entail? Processing the get request and packaging the entire dataset into a REST object right? Or is it a more complex API that lets you run queries against the dataset? For that matter wouldn’t downloading a CSV also have to be packaged into a REST object?
the rest API provides parameters for filtering & pagination. all of that is unnecessary. it's a few hundred MB tops . CSV , Bigquery , anything is better than running REST on EC2
On that note, what would the processing entail? Processing the get request and packaging the entire dataset into a REST object right? Or is it a more complex API that lets you run queries against the dataset? For that matter wouldn’t downloading a CSV also have to be packaged into a REST object?