[petewarden]

Probably a bit of both - I'm no SQL wizard, and I'm indexing 300 million+ Twitter messages and storing the @reply relationships (or more generally analyzing massive communications stores, eg an organization's email server).

I'll try to put together some reproducible code, but for example I had a relationship table that once it grew into millions of entries, took noticeable wall-time to simply access via a primary key. Switching to InnoDB after spotting a note in the memcache FAQ that it supports much faster primary key fetches fixed that, but before that I was looking into the black box with no good leads on fixing it.

[falien]

Sounds more like you needed to read a book about mysql + database design. Its good to come up with workable solutions using what you know. Its better to figure out what you need to know for a great solution and go learn that.

That's my philosophy anyway. And I would imagine reading just enough to know whether you really should start from scratch again, or read much more to grok the required intricacies, would be worth it.