Yep, the up vector is not necessarily orthogonal to the direction vector (which is also called “look at” vector in OpenGL [1]). Another approach would be to set
We've already defined z and x by that stage, and we need y to be a unit vector perpendicular to both. So y can only be z.cross(x) (or x.cross(z) for mirrored).