I'm not the person you replied to, but I came from using VMware products for 12 years, to using Proxmox this last 1.5 years.
These are my impressions.
First of all it's a very competent product, mainly thanks to Ceph making it HCI. Without Ceph, I'm not sure what we would do.
It's as effective as you design it, make sure to separate storage and cluster traffic to ensure robustness, and speed. Make sure to use at least 10GbE switch for storage, for fast migrations.
And managing ceph is very important, basically boils down to 1) never let it run out of space, and 2) the more devices you have the easier it is to manage.
Automating against Proxmox definitely is the biggest pain point, and this needs the most work done.
I've spent countless hours, pre-AI, building our automation setup using both Terraform and Ansible. I sort of wish I had tried AI earlier because it does make things easier.
Some things like automating the creation of templates will forever be a complex procedure in Ansible. And I abandoned Terraform completely because the API was too unpredictable for Terraforms strict state, Ansible was a much better fit.
Their AuthZ takes some getting used to, the fact that if you select "Privilege Separation" it countes the user's permissions AND the token permissions, and the token permissions must always be lower than the users.
Templates existing on one node, but taking a unique VM ID across the cluster is also a bit confusing. It means in practice we're always deploying VMs on the same node, before migrating them somewhere else.
So you deploy a new VM from a template, shut it down, take a backup and then restore that backup to your target node. Is all this done with IaC? Ansible? Even the backup part?
Typically I don't touch templates once I do their initial setup, just shut them down and take a backup (can be done using ansible through the PVE CLI)
The backup restore and the VM startup is done through ansible > PVE CLI.
I also have a testing VM that has a "CLEAN" snapshot that I restore to multiple times a day, using ansible > PVE CLI. Once the VM snapshot is restored I turn it back on as well using the PVE CLI
These are my impressions.
First of all it's a very competent product, mainly thanks to Ceph making it HCI. Without Ceph, I'm not sure what we would do.
It's as effective as you design it, make sure to separate storage and cluster traffic to ensure robustness, and speed. Make sure to use at least 10GbE switch for storage, for fast migrations.
And managing ceph is very important, basically boils down to 1) never let it run out of space, and 2) the more devices you have the easier it is to manage.
Automating against Proxmox definitely is the biggest pain point, and this needs the most work done.
I've spent countless hours, pre-AI, building our automation setup using both Terraform and Ansible. I sort of wish I had tried AI earlier because it does make things easier.
Some things like automating the creation of templates will forever be a complex procedure in Ansible. And I abandoned Terraform completely because the API was too unpredictable for Terraforms strict state, Ansible was a much better fit.
Their AuthZ takes some getting used to, the fact that if you select "Privilege Separation" it countes the user's permissions AND the token permissions, and the token permissions must always be lower than the users.
Templates existing on one node, but taking a unique VM ID across the cluster is also a bit confusing. It means in practice we're always deploying VMs on the same node, before migrating them somewhere else.