|
|
|
|
|
by kamens
5094 days ago
|
|
Ah. Well, first of all, we have already gone through the pain of building a pipeline to export the majority of our heavy data analytics to a Hadoop/Hive setup on EC2. So, yes, we only use App Engine's mapreduce in certain cases where it makes sense. However, what I'm specifically referring to in this blog post is the ability to keep relying on App Engine's datastore for the everyday work involved in serving our application (forget the mapreduce stuff) while gaining more flexibility to run non-App Engine pieces of software on the virtual servers without suffering the App Engine-to-EC2 latency pain. A trivial example would be Lucene (right now we have to run it on EC2 and communicate back'n'forth). Another example would be our own memcached servers that we control the size of. |
|