at least with one of my banks customer support centres this wouldn't happen, if you stumble for a split second they shut down the call and tell you to go into a branch to verify your identity, this is pretty annoying...
That's terrible, because it makes using password managers impossible (while on your phone for example, or you simply don't have it open that instant because you didn't know when/if they would ask).
(Anyway, I like the idea of using answers to security questions as hard passwords.)