Say I want to fine tune a Golang specific model. How much $ and effort would I have to put in? Would using this as a base help in any way compared to starting from llama?
Maybe it makes sense to start from llama-code, not llama :D I think golang specific model will not be that different from a multi-language model. But it definitely will work better after fine tuning on your code. Check out refact self hosting docker in a couple of days, finetune will be there soon. It will take you 1 GPU and almost no money )