it's not really an improvement if you have to run your dev environment inside a VM, with the inconvenience of not having access to the files in the host.
From looking over your Vagrantfile it looks like the project files are mounted from the host into the VM so ssh would only be required for tunneling to the db connection? (Although that can also be avoided by exposing a port or using bridged networking).
The biggest issue I've found with this kind of setup is that file change events from the host don't cascade to the virtual machine using VirtualBox's shared folders. NFS handles this better (still seconds delay based on cache values) but that is then a problem for Windows users.
It's been a few years since I switched to using Linux natively so my knowledge on Docker for Desktop Mac mount strategies may be out of date, but at one point docker introduced options such as :cached and :delegated which can have a considerable improvement to container file access speeds. Additionally, I've heard of people using mutagen and nfs for significant improvements, but I've never tried that setup. Here's a long discussion about it: https://github.com/docker/for-mac/issues/1592#issuecomment-6...
I've tried it all and the only real solution is a Linux VM without shared folders, which I'm fine with. The project I booted is used by other colleagues so I tried to make it "generic" and then yes, vagrant copies the file.
As you said (and others in comments) bidirectional syncing with docker-rsync, or using NFS, is one way to have files synchronising but it always ends up being a pain.
I did use cached and delegated but it's nowhere near close to the ideal solution: removing macOS of the equation. At least until Docker find a real fix for those performance issues, if anytime.