Depending on how complex or time consuming some of these intermediate stages can be, it can be helpful to explicitly cache from them during the build process [0].
If you opt-in to using the experimental BuildKit support that Docker has, you can also do some neat tricks like potentially skipping unused layers [1], as well as building layers in parallel based on their dependency hierarchy [2].