No, you can't initiate a session with someone you know the number of, it demands access to contacts so "you can stay in contact with your friends". The usual weasel words.
It was the case before, but I can do it now. It says "enable contact access to make it easier" but I can also just punch in a number and start chatting... Contact access is off.