[mytailorisrich]

You are answering your own question, I think. Yes you want to keep emails unless obviously you think they may be incriminating.

The usual advice, though, is obviously not to put in writing what you don't want to be found later...

[swiftcoder]

It's hard to tell a priori what will be incriminating, especially once there is more than one person's email involved