[crznp]

You could still make the phone a primary device and allow it to perform the key agreement and pass control off with a QR code, but that is complicated and leaves open the question of who is allowed in this conference.

So perhaps you just give up on persistent identity: just have an unencrypted waiting room, the organizer and their delegates can approve people in the waiting room to enter the encrypted conference.

[GordonS]

Do you mean kind of like how authentication is sometimes handled on input/UI constrained devices (e.g. TVs), where a message could be played to callers, asking them to enter a one-time code at a particular website?

On the face of it, this could work quite well for most people.