[AzN1337c0d3r]

Most workstation class laptops (i.e. Lenovo P-series, Dell Precision) have 4 DIMM slots and you can get them with 256 GB (at least, before the current RAM shortages).

There's also the Ryzen AI Max+ 395 that has 128GB unified in laptop form factor.

Only Apple has the unique dynamic allocation though.

[the_pwner224]

Yep, I have a 13" gaming tablet with the 128 GB AMD Strix Halo chip (Ryzen AI Max+ 395, what a name). Asus ROG Flow Z13. It's a beast; the performance is totally disproportionate to its size & form factor.

I'm not sure what exactly you're referring to with "Only Apple has the unique dynamic allocation though." On Strix Halo you set the fixed VRAM size to 512 MB in the BIOS, and you set a few Linux kernel params that enable dynamic allocation to whatever limit you want (I'm using 110 GB max at the moment). LLMs can use up to that much when loaded, but it's shared fully dynamically with regular RAM and is instantly available for regular system use when you unload the LLM.

[wilkystyle]

What operating system are you using? I was looking at this exact machine as a potential next upgrade.

[the_pwner224]

Arch with KDE, it works perfectly out of the box.

I configured/disabled RGB lighting in Windows before wiping and the settings carried over to Linux. On Arch, install & enable power-profiles-daemon and you can switch between quiet/balanced/performance fan & TDP profiles. It uses the same profiles & fan curves as the options in Asus's Windows software. KDE has native integration for this in the GUI in the battery menu. You don't need to install asus-linux or rog-control-center.

For local AI: set VRAM size to 512 MB in the BIOS, add these kernel params:

ttm.pages_limit=31457280 ttm.page_pool_size=31457280 amd_iommu=off

Pages are 4 KiB each, so 120 GiB = 120 x 1024^3 / 4096 = 31457280

To check that it worked: sudo dmesg | grep "amdgpu.*memory" will report two values. VRAM is what's set in BIOS (minimum static allocation). GTT is the maximum dynamic quota. The default is 48 GB of GTT. So if you're running small models you actually don't even need to do anything, it'll just work out of the box.

LM Studio worked out of the box with no setup, just download the appimage and run it. For Ollama you just `pacman -S ollama-rocm` and `systemctl enable --now ollama`, then it works. I recently got ComfyUI set up to run image gen & 3d gen models and that was also very easy, took <10 minutes.

I can't believe this machine is still going for $2,800 with 128 GB. It's an incredible value.

[xnzakg]

You may wanna see if openrgb isn't able to configure the RGB. Could even do some fun stuff like changing the color once done with a training run or something

[rwmj]

I use openrgb to turn off all the RGB crap on my desktop machine. Unfortunately you have to leave openrgb running and it takes a constant 0.5% of CPU. I wish there was a "norgb" program that would simply turn off RGB everywhere and not use any CPU while doing it.

[parlortricks]

Yeah, second the norgb option. Even more annoying when openrgb just randomly hangs scanning devices and now im stuck with rainbows i cant turn off!

[jrgd]

Brilliant!

[wilkystyle]

Really appreciate this response! Glad to hear you are running Arch and liking it.

I've been a long-time Apple user (and long-time user of Linux for work + part-time for personal), but have been trying out Arch and hyprland on my decade+ old ThinkPad and have been surprised at how enjoyable the experience is. I'm thinking it might just be the tipping point for leaving Apple.

[jrgd]

I just did! Warmly encouraging you to try it out! Managed to put Omarchy on an external ssd on my old macbookpro 2019; rarely booting in macos now. Long time i haven’t enjoyed using a computer SO MUCH!

[lambda]

> Only Apple has the unique dynamic allocation though.

What do you mean? On Linux I can dynamically allocate memory between CPU and GPU. Just have to set a few kernel parameters to set the max allowable allocation to the GPU, and set the BIOS to the minimum amount of dedicated graphics memory.

[AzN1337c0d3r]

Maybe things have changed but the last time I looked at this, it was only max 96GB to the GPU. And it isn't dynamic in the sense you still have to tweak the kernel parameters, which require a reboot.

Apple has none of this.

[the_pwner224]

Strix Halo you can get at least 120 GB to the GPU (out of 128 GB total), I'm using this configuration.

Setting the kernel params is a one-time initial setup thing. You have 128 GB of RAM, set it to 120 or whatever as the max VRAM. The LLM will use as much as it needs and the rest of the system will use as much it needs. Fully dynamic with real-time allocation of resources. Honestly I literally haven't even thought of it after setting those kernel args a while ago.

So: "options ttm.pages_limit=31457280 ttm.page_pool_size=31457280", reboot, and that's literally all you have to do.

Oh and even that is only needed because the AMD driver defaults it to something like 35-48 GB max VRAM allocation. It is fully dynamic out of the box, you're only configuring the max VRAM quota with those params. I'm not sure why they choice that number for the default.

[lambda]

You do have to set the kernel parameters once to set the max GPU allocation, I have it set to 110 GiB, and you have to set a BIOS setting to set the minimum GPU allocation, I have it set to 512 MiB. Once you've set those up, it's dynamic within those constraints, with no more reboots required.

On Windows, I think you're right, it's max 96 GiB to the GPU and it requires a reboot to change it.

[numpad0]

Intel had dynamic allocation since Intel 830(2001) for Pentium III Mobile. Everything always did, especially platforms with iGPUs like Xbox 360.

Only Apple and AMD have APUs with relatively fast iGPU that becomes relevant in large local LLM(>7b) use cases.