[valgaze]

Fine-tuning is such a dangerous phrased bc it sounds perfect.

“We’ll just fine-tune based on our (we think valuable + special) data”

Fine-tuning doesn’t enhance the model w/ new “knowledge” but a new narrowly-defined task

One other “cost” to consider is fine-tuning a 3rd-party model means if that foundation model changes or goes away that effort/cost needs to be repeated

[JamesBarney]

My understanding of fine-tuning is 95% of the work could be re-used between different foundation models.