I know close to zero about this so bear with me, but isn't there a cost to reading the models off disk and deciding if they need to be rendered? Memory bandwidth isn't free.
Models are Kb in size and insignificant to textures though. It's common practice to load up detailed models and let game engine dynamically adjust LOD. It doesn't hurt performance much at all.