The reason the Go developers cited is fast compile times, as well as the belief that LLVM is "too big". I don't agree with these: LLVM compile times are fine for ahead-of-time compilation, and LLVM is big because it does important things.
I do have mixed feelings about LLVM for safe GC'd languages, though. LLVM is full of undefined behavior, and its support for precise moving tracing GC is not widely used. So I can definitely sympathize with not wanting to use LLVM for Go, but not for the reasons they cited.
What constitutes 'fine' depends on workflow, size of project, machine horsepower, and personal preference. Some projects require a bit more compile -> experiment -> change -> compile -> experiment -> change than others.
I do have mixed feelings about LLVM for safe GC'd languages, though. LLVM is full of undefined behavior, and its support for precise moving tracing GC is not widely used. So I can definitely sympathize with not wanting to use LLVM for Go, but not for the reasons they cited.