[philsnow]

found https://utdemir.com/posts/ann-distributed-dataset.html , which makes me think that kind of distributed approach could be workable for channable (the default existing backend spins up lambda workers and stores intermediate results in S3), but you'd need to write a different backend that farms out work to whatever worker machines you have available

[ghostwriter]

The distributed backend part could be done via a lower-level set of distributed primitives from https://hackage.haskell.org/package/distributed-process