I use Swift professionally and have written some Rust for fun, and like both languages a lot. I find Swift to be faster to write, not least because it's easier to manually prevent retain cycles than to write borrow-checker approved code. It's also much easier to leak memory and write unsafe threaded code in Swift, so there's a trade-off there.
I'm sure if I wrote more Rust I'd get faster at it, though I do think it has a fundamentally more complicated programming model.
For the use-case here, on a server, I think the answer is currently no. We're writing our back-end in Rust, but tried Swift first (fewer languages in the stack!), but there were just too many gotchas.
Personally, given the option, I prefer Rust over Swift. I've written a decent, if not huge, amount of both (notable, I wrote an Avro-to-Swift code generator in Rust), but written more Rust, so take this with a grain of salt.
I've found that Swift's generics have some edge cases that cause problems; there's some rough edges around build that are better in Rust; dependencies are much easier in Rust; and I like Rust's traits over Swift's protocols. That said, protocols in Swift are very cool and flexible, and I think Swift-the-language is very well designed. And of course, for macOS UI or iOS development, I'd go with Swift for the obvious reasons.
The xi editor is mostly written in Rust, but wants native UI on each platform, and therefore uses Swift for the UI on macOS. This contains an answer to your question, but also, something more subtle: that it doesn't have to be either/or.
I'm sure if I wrote more Rust I'd get faster at it, though I do think it has a fundamentally more complicated programming model.