[aliljet]

I'm super duper curious if there are ways to glob together VRAM between consumer-grade hardware to make this whole market more accessible to the common hacker?

[rerx]

You can, for instance, connect two RTX 3090 with an NVLink bridge. That gives you 48 GB in total. The 4090 doesn't support NVLink anymore.

[mk_stjames]

You actually can split a model [0] onto multiple GPUs even without NVLink, just using the PCIe for the transfers.

Depending on the model the performance is sometimes not all that different. I believe for solely inference on some models the speed difference may barely be noticeable, where for other training activities it may make 10+% difference [1]

[0] https://pytorch.org/tutorials/intermediate/model_parallel_tu...

[1] https://huggingface.co/transformers/v4.9.2/performance.html

[koheripbal]

> The 4090 doesn't support NVLink anymore.

Are you sure about that?

[homarp]

that's what the press said: https://www.tomshardware.com/news/gigabyte-leaves-nvlink-tra...

[bick_nyers]

I remember reading about a guy who soldered 2GB VRAM modules on his 3060 12GB (replacing the 1GB modules) and was able to attain 24GB on that card. Or something along those lines.