Not true if there’s enough competition that you need more resources for a bigger model in 2-3 years. Anecdotally, it seems like SOTA model training costs are rising much faster than computer cost is falling.
maybe, but model quality doesn't scale linearly with model size. The performance/dollar metric is more important, and that definitely will decline over time.
Personally, I have a boots trapped business that uses a transformer model. I'm not worried about supermassive models like gpt3 because it would be way too expensive to deploy for my use case, with marginal additional quality.
Personally, I have a boots trapped business that uses a transformer model. I'm not worried about supermassive models like gpt3 because it would be way too expensive to deploy for my use case, with marginal additional quality.