Cryptography is about finding large prime factors of a large number. That's a much harder problem than just determining whether a specific number is prime or not.
I don’t understand you just say it was about finding factors, okay yeah that determines whether a number is prime. You’re not telling me anything I don’t know, I just disagree with your stance.
You seem to be claiming that the ability to determine whether a number is prime or not is extremely difficult and that it would break (some) cryptography if it were no so. My stance is that (some) cryptography would not be broken unless factorisation of large numbers into two large primes becomes easy.
Can you clarify how you think that some cryptography is broken by a relatively simple test of whether a large number is prime or not?
primality checking is much easier that factoring. it's somewhat unintuitive, but there are deterministic methods of primality testing that don't tell you the factors.
AKS is an example of a deterministic algorithm which works out whether a number is prime or composite and runs in polynomial time in the number of bits in the number.