[MertsA]

That might be an interesting problem for a generative adversarial network. Train it based on some large corpus of faces and then refine it based on searching for the input vector that most closely matches the original image and then just change the pose in that input vector to generate a square shot of the face. In theory this would give you not only some generated face, but some reasonable space of facial features that the discriminator couldn't reject. I.e. beard / no beard if the chin was obscured.