The same way you'd know that Signal isn't keeping a copy of your private key somewhere - people can inspect the clients. Yes, the app could use some sort of really sneaky way of leaking the key such that nobody ever finds it. It would also be possible for the developers to distribute a very specific version of the client that does this only for a very specific region and device combo to minimize the likelihood that this is caught. But it is the sort of thing that would both be unimaginably bad for PR and challenging to keep under wraps.
No. But plenty of people are good at looking at decompiled code. I'm more experienced with Android reverse engineering, so somebody else would need to comment on ios. But dalvik bytecode isn't too far from what people are used to looking at and the decompilers up to java are quite good. The native libs are the only actual challenge and even then there are tons of people out there who have no problem throwing ghidra at a binary and understanding it effectively.
Publishing the source online doesn't fundamentally change the way that interested parties can inspect the client. And the reversing process has the nice property of actually guaranteeing that the thing you are looking at is what runs on the device.
Obfuscation is annoying at best, doesn't do much otherwise. Someone has to follow what happens to the private key through the control-flow graph anyway. (And if some function says innocent_crypto_method2 it still needs to be "audited" anyway.)
If the key ends up used for signing and authenticating messages and for nothing else, then it's sort of safe to say that it's not leaked. (Sure there might be some other part of the code that reads it indirectly, but that's also something that will likely not be named leak_priv_key() :))
The only thing that would help is open source + trusted reproducible builds.