If you need history then still put them on a web server and increment the filenames. Storing large files in a git repo is a misappropriation of the tool. It wasn't designed for that use case.
I mean for me I'd like the convenience of having it all together. Oftentimes it would suffice if I could just store the current version of a binary efficiently in git. Marking it, so that git forgets the previous versions.
Currently I'm using an artifactory for this, but it would be much nicer if this could be integrated.