The mental overhead of doing "safe memory programming" is high already. The difference between Rust and, say, C, is that Rust forces you to do "safe memory programming". C lets you get away with unsafe memory programming.
C lets you deliberately do unsafe memory programming, which is maybe OK if you actually know what you're doing. But C also lets you think that you're doing safe memory programming, when that is not the case.