The Rust code is still only brute force - using suitable persistent acceleration structures you can probably get a 10x again or maybe even 100x, in 2D a kd-tree is really fast for NNs.
So much faster that the allocations for the result will probably be the bottleneck.
"That Rust library will be faster than the corresponding python library" is a useful thing to know here.