You're right I do fail to consider this, as it's not really an alternative to Shor, because annealing is horse shit that nobody can decide the computational complexity of. Not even D-wave thinks it might be. [1]
I'm not spreading FUD; I am correcting misinformation from muppets whose understanding doesn't go beyond press releases. Nobody has yet done a Shor factorization of the number 15; the end, and even if someone's press release says so there is no scalable way of factoring large integers.
Quantum annealing will never be used for factoring prime numbers from large integers, and hasn't even managed to factor 3 and 5 from 15. Even if you click your heels together three times and wish for it really hard, it's not going to happen.
Did you read the nature article, or just the press releases?
Besides, the empirical data shows otherwise. It takes 12 qubits to factor 15. We're up to 53 now.
With quantum annealing, a 20 bit number has been factored with 97 qubits. Not on a real quantum computer yet, of course.
So I have no idea what you are talking about.