Yes, the VM itself is not fast. However, a lot of it is just a thin shim over a C call, so whether you're going to be bottlenecked by it is going to depend on what you're doing.