|
|
|
|
|
|
by fooblaster
723 days ago
|
|
|
All optimizations inevitably end up just being funneled into larger and larger models. The issue with AI is its seemingly endless ability to scale in size for marginal gains in model performance (log(n) scaling essentially). |
|
|
Well of course if you're trying to beat the SOTA, bigger sizes allow for better models, but not everyone is trying to use or train the latest SOTA model, maybe llama-3 8b is perfect for what you need to do and having better optimizations to run it locally is gold.