[oblio]

One second, don't LLMs generally run in VRAM? If you put them in regular RAM, don't they have to go through the CPU which kills performance?

[pxeger1]

The mentioned CPU uses unified memory for its built in GPU / NPU. I.e. some portion of what could ordinarily be system RAM is given to the GPU instead of the CPU

[oblio]

Ah, now I see, didn't know that was feasible in the PC world. Glad that it's becoming an option.