Being able to pull it from a dead phone seems like a huge security issue? Shouldn't storage be encrypted using private material from a secure element? I understand that people here are tech-savvy enough to only store music, etc. on an SD card. But I think a lot of less technically-inclined users would set themselves up for losing private data.
Depends on your threat model and priorities. Historically the biggest thing I wanted to pull off the SD card was encrypted backups, which were in fact encrypted enough that an attacker getting them wouldn't be a serious problem, but which were rather handy to have. (And yes, I can completely push those off-device, but an SD card is a handy middle ground of local, fast, easy, safe from the things I tend to be worried about (mostly, bricking the device), and big enough that dumping 10s of GB on there is fine.)