I guess that it's not a proximity problem. For example, in the first video at 3:08 a gray mountain with snow appears on the upper right corner, and replaces a piece of sky. I think that a big rock was occluding the vision of the mountain in this frame, and the algorithm has to choose a texture from another frame to fill the void, and it made a mistake.
If you watch the technical video, they say that they couldn't use the scene reconstruction for the climbing video as there were too many artefacts. This is why the rendering isn't as good as the others.