[alexvoda]

It would need "to want" "to avoid bad consequences". It is rather circular because of the word "bad" which implies "do not want".

If it is indifferent to all consequences, you are back to square 1, so no, consequences are not the missing magic. Consequences are a function that transform one "want" into a different "want".

[Misdicorl]

You aren't back to square one. Even indifference will result in selection so long as the indifference is demonstrated in different ways by different AI. The ones which have behavior which accidentally maps slightly better to good outcomes will so better. So long as that gets propagated more strongly to the next generation of AIs than average, you get selection for good outcomes without any explicit 'want' function