| this product really deserves the "halo" in the name. very difficult to gauge its fit in the current market. if you want inference, go mac with much higher memory bandwidth. given the price premium here, or the little there is, you might as well. if you want to finetune and experiment, cuda still has the moat and the kit is not much cheaper, if at all, than dgx spark. from personal experience, i had access to amd developer cloud with a fair bit of credits. however, even doing inference outside of their supported use cases (which are often dated btw) using vllm was a pain. in the end, despite great computing potential on paper, i decided to not spend more time than its worth on it. if their enterprise cloud continue to have these grievances, i am not optimistic about this kit. it might be down to skill issue on my end. perhaps if these sell and it gives amd enough motivation to add more software staff in-house, more power to them. otherwise, good article from labs as usual. nice to know that other kits based on this soc are more or less the same (unsurprisingly). |