[modeless]

Yes, it absolutely could. You're right that this configuration is rare. Although people have been putting together machines with multiple 24GB cards in order to split and run larger models like llama2-70B.

[wahnfrieden]

The latest large models are 120B and 100k context such as Goliath and Tess XL