Not saying JPEG hasn't had its share of exploits, but that the format itself isn't an example (like DOC, and PSD) of a format that can be described as "direct to memory" in the same sense. DOC and PSD are in many ways just the in-memory data structures of the apps that create them, written to disk.
"These are binary formats, so loading a record is usually a matter of just copying (blitting) a range of bytes from disk to memory, where you end up with a C data structure you can use. There’s no lexing or parsing involved in loading a file. Lexing and parsing are orders of magnitude slower than blitting."
PSD is much the same - it's why interop with PSDs is such a challenge.
JPEG has to be parsed and decompressed - another source of problems, but not remotely in the same category.