Can we modify this approach to get LLMs that are good at specific programming languages or frameworks? That seems to be where local LLMs could really shine.
LLMs shine through emergent behaviour. Finding an LLM that does Rails doesn't know poetry is like finding a Rails human developer who doesn't have a hobby e.g. basketball. So what if they play basketball? They can code too!
Then it might need a new type of architecture to work. I’m not attached to LLMs. If a new model comes out that can do only the things I want it do it, then great
Alternatively, a modular model with multiple “experts” that I could mix and match for my specific stack
I don’t need the model to know all of the Internet plus 20 different human languages. I just want it to be really good with the stack of the project