Few month ago I tested this,
and it's seems they do not cache. Instead they are just using proxy to enable anonymous download of the image.
But you never it any cache from google directly.
They do cache at least sometimes, and they must for it to work.
If I send you an image URL http://example.com/fH27cAw, and they URL is requested, you must have opened my email (unless something is requesting them regardless).
You'll see that it in fact does get cached by Gmail, but only for a limited time.