[tasqa]

I found proxmox not to be very user friendly growing to such cluster sizes. Proxmox itself has been very stable and supports pretty much anything but the GUI is not that great if you have many nodes and VMS, and the API can be lacking. However, using ceph as a backing store for VM images is pretty easy in proxmox. I have not used the cephFS stuff. I used it in a separate cluster both physically and standalone (not using proxmox integration).

So RBD is easy, S3 is somewhat more complicated as you need to run multiple gateways, but still very doable. The FS stuff also needs extra daemons, but I have not yet tested it.