Hacker News new | ask | show | jobs
by roguefort 1164 days ago
Unless you are an established organization. I don't see why anyone will move away from APIs or finetuning. Custom models are going to get fewer and far between.
1 comments

That's another problem with that article. The gap between API and fine-tuning is much smaller than the gap between fine-tuning and developing a foundation model. I would look at BloombergGPT as an example

https://arxiv.org/abs/2303.17564

here you have a company which can make a document collection about the same size as "The Pile", add that to "The Pile" and train a model based on that. They're not just a big company but they are in the information business so it is clear that it's worth it to them.