[krobertson]

We looked at RDS and had a call with some of their engineers, but we basically had our EC2 + raid'd EBS set up almost the same as they did, all best practices already being done.

Since RDS really is EC2 + EBS, they couldn't provide any real assurances it performed better than our own installation.

We ended up moving off of AWS as a whole. After several discussions about how we can continue to scale, the ultimate answer was without AWS.

EC2 is great for distributed stuff, but when need something that is heavy IO, for instance, it is a big problem. Scaling it ends up costing more to work around AWS's performance problems than to go elsewhere.

[ketralnis]

Yeah they have a few products (e.g. EMR, RDS) where they charge by the instance anyway so you're just paying them by the hour for the five minutes it would take you to set up the server once

[adpowers]

Hmm. I think you underestimate the effort that is spent on those two. RDS has really good replication which is really hard to configure and set up yourself. And having configured Hadoop I know it takes more than 5 minutes :) Perhaps Whirr makes that easier. Also, EMR's Hadoop is tuned to work really well with S3, which you don't get with stock Hadoop (or even with Cloudera's).

[SkyMarshal]

Did you guys find a better cloud service, or did you roll your own in a datacenter somewhere?

[krobertson]

We went with a managed hosting provider who built us a private cluster. Basically a private cloud. But that way we could get a dedicated SAN and move our DB servers out to dedicated boxes with whatever disk configuration we desired.

[SkyMarshal]

Interesting, I wonder if this becomes a trend as other startups and cloud customers discover these limitations and look for more custom solutions.

[jedberg]

Which provider? Can you email me? [e-mail].