You should always be able to build a Go 1.x compiler with just the 1.4 tool chain binaries. We have committed to sticking to the Go 1.4 language and libraries for the compiler tool chain.
> Are patches tested against 1.4 tool chain binaries?
The builders build the tool chain with Go 1.4, so the build dashboard will show failures if a patch incompatible with 1.4 is submitted. (We have pre-commit trybots that do the same.)