Just be careful with a bastion host that you have a fallback. If anything happens to the bastion host and you don't have access to that IP, your VPS is basically lost. If you use a bastion host, I'd at least have two.
This shouldn't be a big problem as you can use configuration management such Ansible to get around. In case of bastion loss you can simply create a new instance and run the script to recover your instances.