[sireat]

Interesting problem.

I imagine the juicy part is that there is no clear cut formula for determining the rotation from the area needed.

Thus the task is to come up with a fast approximation algorithm to work backwards.

So brute force way:

  If area_needed > area_cast:
    rotate_cube(delta)
    delta=/2
  else similar rotation on -delta

Now, my 3d geometry is rusty, so maybe rotation on 2 axis is enough but still optimizing 2 arguments is not going to be easy. Maybe you can max one rotation then max 2nd one.