[undefuser]

With that amount of memory can you run 4-bit DeepSeek 4 Flash? It is way more efficient in the KV cache department so may be worth a try

[CamperBob2]

I haven't looked into DS4 yet but based on antirez's results on 128 GB Macbooks, it shouldn't be a problem to run it on a pair of RTX6000 Pros.

Also see https://www.reddit.com/r/LocalLLaMA/comments/1sv649s/to_run_... .