Refactoring can be its own source of problems. I've seen far too many refactorings that merely exchanged one mess for another. I'm a bit sympathetic to a manager who is skeptical of refactorings.
Definitely, I've seen those kind of refactorings too, but what I mean is that one isn't allowed to fix the architectural problems that make it hard or impossible to implement what's requested. The existing design didn't anticipate certain things which are now impossible to implement without compromising the architecture.