You can take memory from the VM. I've just tested it on KVM, dropping the current memory allocation for a vm, and reraising it. Shows up in free in the vm instantly.
Well it sounds like that is possible, but is there any way for the VM to indicate to the hypervisor that there is unused memory? Or for that fact, that it needs more? That's the critical missing piece, unless there's an API I'm unaware of.