The most often encountered thing that I find Rust frustrating is how much the language changed over time--lots of random answers and posts that are outdated floating around somewhere that I have to throw out because the language changed.
It's just incredibly young at this point, and still in a phase where making breaking changes for the sake of the language vision is acceptable. Swift has been in the same boat. I'm sure that will change in the near future. Luckily both languages have great compilers that catch backwards-incompatible issues.