without looking at the code, I can guess that things like occlusion and visibility calculations are a lot harder when you are within a city than when you know the entire city is in the same direction relative to the camera.
I don't think LOD optimisations are enough when you are in the middle of a solid with complex holes in it, but that's a detail (no pun intended) - there are doubtless lots of other techniques you can apply. but my larger point was whatever optimisations you make, performance when looking at the entire city from the outside is likely to be better than looking at some part of it from the inside.
https://en.wikipedia.org/wiki/Level_of_detail_(computer_grap...