[pondruska]

Yes, in nutshell, this is it :-) But knowing depth information is not enough - you must also very precisely know the position of the camera the individual pictures were taken from. Computing both depth information and position of the camera is in fact a very challenging task.