[wvenable]

MySQL is a good product, but it does have some significant failings when it comes to getting top-notch performance out of indexes.

Digg's description of their entire setup seems a bit unusual -- from how they've defined their tables to their query methods. It seems at least somewhat likely that they were not making optimal use of their technology.

I'd be curious what the traffic/content difference between Digg and Stackoverflow is. Stackoverflow uses an architecture very similar to what Forbes proposes here and they have plenty of capacity on rather unremarkable hardware.

[jbellis]

I would bet that SO has several orders of magnitude less traffic than Digg.

[ratsbane]

Looking at Alexa's numbers for a rough comparison of pageviews it looks like Digg gets about 10x as many as SO:

http://www.alexa.com/siteinfo/stackoverflow.com#trafficstats

[gill_bates]

So while SO runs on a minimal amount of closet-room hardware efficiently and without incident, Digg has heroics and clusters and scale-outs and all sorts of drama to handle just 10x the load? I'm not sure what angle you were coming from, but that makes Digg look like clowns.

[skorgu]

Not saying that Digg aren't clowns but the traffic balance makes a huge difference. It's not implausible that Digg has an enormously bigger fraction of writes to reads than SO does which changes the scaling dynamics considerably.

Edit: To expand I'd imagine SO serves a lot of static "how do I" google hits for logged-out users which is about as easy to serve as it gets. Digg puts a higher emphasis on the logged in experience and commenting which reduces the ability to cache.