[empath75]

Is this for just like auto complete, because you are not going to get anything very useful out of a code-only training set.

[JohannaAlmeida]

Yeah auto complete is an amazing use case. I needed a small model that used transformers , could fit on my weak consumer GPU .

So i needed to make fundamental arquitecture changes .Do some KV cache tricks.

And then prove the new arquitecture was faster with benchmarks and perplexity was acceptable.

[bigbadfeline]

Well, coding is a kind of extended autocomplete - I prefer that way of working because I don't like the mess created by LLMs when you let them work on their own. Smaller models, specialized on a single language, make a lot of sense.

[altruios]

I think it's more a proof of concept: locally trained. It would take lots of resources/time to train something non-trivial.