[mvanbaak]

every package system (apt/yum/pkg/whatever) is distributing binaries. So yeah, the upstream project can be open source, but there is 0 guarantees that the binary I install on my system is the exact same binary as I would get if I build the source myself (and this does not even touch on the subject that compilers can add weird stuff as well)

Sure, it's better than closed source, because at least you have the possibility to check all this. In practice though, we outsource this responsibility to the package maintainer of the package system we use.

[em-bee]

there is 0 guarantees that the binary I install on my system is the exact same binary as I would get if I build the source myself

not true, for years there are efforts in various distributions to make package builds reproducible. there are ways to build a package from source that allows you to get the same results and verify it.

we outsource this responsibility to the package maintainer

which is the point. i trust the package maintainers to do a better job at that than myself.