Seems like so much more work than "just" paying for https://huggingface.co or whichever other neocloud who already did all the setup for you and just waits for your credit card per minute/seconds/token.
Doubt this kind of workloads would agree to send data then to a cloud of randos devices, precisely when cloud providers to certify they aren't looking at clients data (Customer-managed encryption keys, CMEK).