[skrause]

brew supports pre-compiled binary packages (what they call bottles) only when you use the default install location /usr/local. If you use any other prefix brew will always compile all packages on your system, which might take a long time depending on the performance of your system. So using /usr/local saves time and energy since all brew needs to do is to download and install binary packages.

[__david__]

Actually, many bottles are marked as safe to install anywhere, so even if you don't use /usr/local you will still get a lot of things installed via bottles (but not everything).