[ALittleLight]

I would think a clear sign of overfitting is if your training loss is decreasing while your validation loss is increasing. That tells me your model is learning to perform better on the training data at a cost of performing worse on more general data.

Is that happening when you train these micromodels? If not, I have a hard time seeing how it's overfitting because the model is still performing well for the data you train it on and use it on. If that is happening, then I don't see the benefit of it. A model that wasn't overfit would just do better at the task of collecting additional training data.

I think the approach you're talking about makes sense - create a simple model rapidly and leverage it to get more training data which you can then use to refine the model to be better still. I just don't think the term "overfitting" describes that process well - unless I'm misunderstanding something.