[edb_123]

I just recently got into experimenting with local LLMs when I had anyway (for non-LLM reasons) built myself a new desktop system with Intel Ultra 270K-Plus and RTX 5080. With 64GB system RAM and 16GB VRAM. Relatively speaking a high-performing and low-to-moderate cost system.

I wasn't really expecting much from these local open weight models neither when it comes to speed or "intelligence", but my preconceptions were quickly put ashame when I got ollama up and running and pulled my first model. I get a consistent 117-128 t/s with Gemma4:26b-a4b without any tuning (just the default settings), which was much faster than I had expected. Can't wait to dive deeper into this, especially with Qwen3.6 models.

Does anyone's have experience adding a 2nd Nvidia GPU of the same generation but different (slower) model in the same system? Will it give a major boost with larger models, or will the slower card just be a bottleneck? I have an unused RTX 5060 Ti 16GB that I'm considering to install alongside the RTX 5080, but it would necessitate removing some other hardware, so I haven't bothered yet.

[skolos]

I'd say adding another 16Gb gpu would be worth it - you'd be able to run larger model/larger context all within gpu's. It would give you more options of what you can run fast. Your current model probably doesn't run completely from GPU (depending on quants I don't think you can squeeze Gemma4:26b into 16Gb vram), so you already have some layers running on gpu and some on cpu. If you add another gpu you might be able to move all layers to vram which should speed up things for you. The layers calculations happen on whatever gpu's it sits, so the layers that are already on your rtx5080 would compute same, but the layers that currently your cpu handles will be computed with faster vram/compute of rtx5060.

[edb_123]

Thanks! I'm seeing a 10/90 split between CPU/GPU with gemma4:26b, so I guess there's at least something to win there by adding the other GPU. And perhaps something to win by connecting the monitor to the iGPU instead to free up VRAM, from what I gather.

Just in case someone should be interested in how a consumer PC setup like this performs, still using only 1x RTX 5080 + 64GB system RAM and Intel Ultra 270K-Plus; I tested Qwen3.6:35b-a3b now (using ollama and default settings) and I'm getting around ~86 t/s. The lowest I've seen so far is 70 t/s. The CPU/GPU split with 35b is 39/61% (with 4K 165 fps monitor connected to 5080, so there's probably some room for optimization here by moving it to the iGPU).

Best thing is that this setup is basically dead silent (it could, hypothetically speaking, be running in my bedroom just fine, and I'm a light sleeper).