OP, I think you’d really enjoy the book “A Philosophy of Software Design” by John Ousterhout. It discusses this topic at length. With respect to reducing complexity, you pretty much drew the same conclusion he did.
I came into this thread thinking of recommending this book. Are there any other books like it that discuss managing complexity, maybe not just for software engineering?