[tyvkiuiyi]

Depends on utilisation. If you own a cluster and are able to keep it busy 24x7 then it pays off pretty quickly. Amazon have to have enough spare idle capacity to handle unexpected customer loads and make a profit - you are paying for this.

[sophacles]

Good point. I wonder at what % utilization a cluster beats EC2 in terms of $/computation. Actually I'm sure there are all sorts of variables, and it becomes an optimization problem, but I think it would be cool to see an analysis of this in terms of $, computation power (time/jobsize or something), etc.

[tyvkiuiyi]

Custer scheduling is a huge area. I used to work on MPI clusters and it is an art to balance CPU, Bandwidth, propagation time to pick the optimum number of processors for a particular algorithm.

Especially on commodity ethernet based MPI, it doesn't do broadcast so shipping a Gb common dataset to 64nodes can take a lot longer than actualy doing the calculation.

[sophacles]

Strange -- I always just sort of assumed that since they are making big clusters, they could spend the extra $$ for a good multicast switch, and that MPI did ip multicast. (a quick googling shows me to be wrong...).

[jacquesm]

I'm pretty sure a lot of people would donate for a statue in your likeness if you solved that.

[easp]

My understanding was that a lot of research clusers actually have pretty low utilization. I believe the cloud computing whitepaper David Patterson put out in the spring had I data on that.