[fbdab103]

Are there instructions for this distributed inference somewhere? Can I do this out of the box with llamacpp or similar?

[londons_explore]

Don't think so. I suspect it would require quite in-depth surgery of llamacpp to add in the ability to send activations over the internet and pipeline stuff to keep all the cores busy.