|
|
|
|
|
by whimsicalism
1069 days ago
|
|
What? You work on the weights - you just do it using tools like the optimizers, etc. You release your weights, others can build on top of that, fine tune it in different ways, produce new weights they can share with others. Seems very OSS-y. I feel like there is some semantic nitpicky point being made here that is completely going over my head. |
|
For all practical purposes, if you are part of the team who released the LLMs, you would be writing and modifying the code of data processing, of the model, and of the training process. Those should be considered source code.
And we do have the model, which is pretty Oss-y, and which is why we can fine-tune the weights. But from a broader perspective, it's not fully Oss-y, because we don't have the code for anything else. There's no way to change, for example, how the training is done in the first place.