Sure, you can see the code / source / raw binary. But actually analysing and verifying every part for security or privacy concerns - bearing in mind you have to consider all the possible interactions this code can generate - and that's before you even get to the hardware interactions - is still beyond 99% of developers, never mind normal people.
I was speaking to "There is no privacy if you can't inspect it and verify that it's upholding its promises" actually, and I probably agree with you, as your point applies just as much to open source.