It is narrowly true that the libressl devs are not cryptographers, but that's also quite misleading. Lots of bugs, like say... this one, are not crypto related.
While that's certainly possible, it's an extraordinary claim because it flies in the face of generally accepted beliefs. If your coworker was Bruce Schneier, I would pay close attention to his explanation. If they were your standard issue sysadmin types, then I'd want to know:
1) Why they believe so,
2) Why they haven't filed security advisories to advise the rest of us, and
3) Why you don't hear about banks being wiped clean because crackers were able to bypass SSH's security measures.
It's possible they're right, but as with all extraordinary claims, the onus of proof is on the ones making them.
2) I tried to explain millions of people around the world rely on it and use it. I argue it's probably safe (within reason - obviously the weak point is the private key file).
These were also the guys who refused to install packages we asked for from the community RedHat repository claiming security vulnerabilities but then they just admitted they installed some packages from there for their own use for puppet and other things they do.