[prashp]

I think your question should be "Will pretrained generative models replace training application-specific models from scratch?" instead of your focus on supervised learning.

The model you mentioned use supervised learning, and fine tuning also requires supervised learning (i.e. the text captions associated with the images).

[baptiste1]

Yes, true. Indeed, your phrasing is better! I am not sure however if I can change the title now. I will keep your comment in mind for the future.