|
|
|
|
|
|
by cosmie
2741 days ago
|
|
|
> Google Analytics is much faster, responds in a few hundreads milliseconds. Are you referring to their reporting API, or their collection endpoint? The collection endpoint is certainly fast to respond, but the actual reporting API can be quite slow depending on what you're trying to get from it. > What did you use dataflow for? How did you get data from end points and insert them into bigquery? Using streaming inserts? I'm not the parent, but I've created setups like what was mentioned. It sounds like they hosted the collection endpoint on AppEngine, then used DataFlow for streaming the data into BigQuery. Potentially using a Pub/Sub topic to queue up for DataFlow, since that has native integrations with DataFlow and even has a template available to support it[1]. [1] https://cloud.google.com/dataflow/docs/guides/templates/prov... |
|
|