Hacker News new | ask | show | jobs
by Imustaskforhelp 158 days ago
Well if someone is gonna use Modal exactly for GPU purposes then I guess its okay but anything compute related just feels like it would have some issues performance wise

So I can agree that perhaps Modal might make sense for LLM's but they position themselves as sandbox including something like running python code etc. and some of this may be more intensive in workflows than others so I just wanted to point it out

Fly.io uses firecracker so I kinda like firecracker related applications (I tried to run firecracker myself its way too hard to build your own firecracker based provider or anything) and they recently released https://sprites.dev/

E2B is another well known solution out there. I have talked to their developers once and they mentioned that they run it on top of gcp

I am really interested in kata containers as well because I think kata runs on top of firecracker and can hook with docker rather quickly.
1 comments
amitprasad 157 days ago
If you're not looking for GPU snapshotting the ecosystem is relatively mature. Specifically, CPU-only VM-based snapshotting techniques are pretty well understood. However, if you need GPUs, this is a notoriously hard problem. IIRC Fly also was planning on using gVisor (EDIT: cloud-hypervisor) for their GPU cloud, but abandoned the effort [1].

Kata runs atop many things, but is a little awkward because it creates a "pod" (VM) inside which it creates 1+ containers (runc/gVisor). Firecracker is also awkward because GPU support is pretty hard / impossible.

[1] https://fly.io/blog/wrong-about-gpu/

link
Imustaskforhelp 157 days ago
Ohh this makes sense now. Firecracker is good for compute related workflows but gvisor is more good for GPU related workflows, gotcha.

For my use cases usually, its Firecracker but I can now see why company like Modal would use gvisor because they focus a lot (and I mean a lot) on providing gpu access. I think that its one of their largest selling points or one of them, for them compute is secondary customer and gvisor's compute performance hit is a well worth trade off for them

Thanks for trying to explain the situation!

link