LLMs have been used to scan binary blobs for exploits already. What would be more effective is a system designed with multiple layers of security so any one exploit is largely useless.
They would have to have access to and scan your individual binary. You'd have to describe how you can write a system with multiple layers of security generally for most problems, because I don't see that as being possible.