Hacker News new | ask | show | jobs
by jimktrains2 2741 days ago
> Google Analytics is much faster, responds in a few hundreads milliseconds.

GA stores summary tables for each day for the basic values. If you have a large site and request segments or anything that's not in the summary tables, it can be quite slow.

Also, BigQuery is multi-tenant. GA would have dedicated instances.

> What did you use dataflow for? How did you get data from end points and insert them into bigquery? Using streaming inserts?

cosmie pretty much got it. AppEgnine collected. DataFlow sessionized and some other processing (geoip lookup, filtering, &c). BigQuery stored.

I actually had AppEngine dumping into Cloud Datastore, but I also experimented with PubSub and also using Cloud Storage access logs.