[backes]

I strongly agree with you on the first point. OpenCV provides calibration primitives which look like they'd solve the problem easily and this gives you false confidence. In my experience, they're very low-level and are not more than a wrapper around the optimization routine. You need to implement most things from scratch, such as correct exposure, checking for blurred frames, verifying the extracted checkerboard,... It's weird that everyone needs to re-invent this process.

Can you be more specific on the second point? Once you have the intrinsics, it's trivial to project them to an (undistorted) image.

[amelius]

Regarding the second point, I'm projecting onto the distorted image (basically the image that I get from the camera sensor) because I don't want to undistort the image for performance reasons. My problem is that the transform is basically undefined just outside the viewport. Maybe I should simply do another check to see if the transformed points make sense. But it breaks my assumption that the inverse-of-the-inverse of a transform is the identity.