[jarcoal]

You’re not missing anything, this is definitely possible to do.

That said, encrypting every message with everyone’s public key doesn’t scale, so instead you’d likely want to generate a symmetric key for the group and as new users are invited the inviter would encrypt the group key with the invitee’s public key. That makes it trivial for new users to view history, and as new chats are posted they only need to be encrypted with one key.