Non-source files should indeed never be in the VCS, but source files can still be binary, or large, or both. It depends on how you are editing the source and building the source into non-source files.
Also, some source files that could otherwise be treated as text⁰ end up effectively being binary blobs because tools don't write them in a stable order, which makes tracking small changes difficult because you can't see that they actually are small changes. A number of XML formats¹, and sometimes JSON & others, have this issue too.
----
[0] for the purposes of change tracking and merging
[1] Stares aggressively at SSIS for its nasty package file format² and habit of saving parts of it in different orders apparently randomly so updating the text of an annotation can completely rearrange the saved file
[2] far from the only crime committed by SSIS I know, but one occasionally irritating enough to mention
Possibly, though I might be concerned that the format has ordering oddities that it is unexpectedly sensitive to. Unlikely, but given how many other oddities DTS/SSIS has collected over the years I'd not be surprised!
Also, we weren't using Git in DayJob at the time we were actively developing with SSIS (maybe VSTS had an equivalent we could have used?), and we are now acting to remove the last vestiges of it from our workflows rather than spending time making it work better with them!
----
[0] for the purposes of change tracking and merging
[1] Stares aggressively at SSIS for its nasty package file format² and habit of saving parts of it in different orders apparently randomly so updating the text of an annotation can completely rearrange the saved file
[2] far from the only crime committed by SSIS I know, but one occasionally irritating enough to mention