[joshspankit]

This might be silly because of how few people it benefits, but could it be broken up on to multiple 8GB cards on the same system?

[modeless]

Yes, it absolutely could. You're right that this configuration is rare. Although people have been putting together machines with multiple 24GB cards in order to split and run larger models like llama2-70B.

[wahnfrieden]

The latest large models are 120B and 100k context such as Goliath and Tess XL