It shouldn't be too bad as long as you keep in mind that it is a reference to an item in that slice, so whatever that slice is pointing to needs to stick around as long as you're using elements from it. I don't often encounter borrow checker issues anymore, because once you program Rust long enough you know what things need to live for what lifetimes, and you architect your "larger programs" around that.
The borrow checker ensures that you either do it right or you find another way that is safe.
For example it may be totally fine to help allocate a result, not all programs need to be optimized to the bone. Just return a boxed value. If you need to share it, wrap it in an Rc or Arc.
One problem I see with it is that rust chooses to make the most efficient way of programming also the most "simple looking", and so it lines up incentives in such a way that people will unnecessarily try to avoid using an extra Arc just because it looks like unnecessary clutter.
When you learn to embrace your boxing you'll learn that the borrow checker is not your enemy.