[NitpickLawyer]

> Can I actually add "features" to the model?

Yes. You can use a number of libraries to add, mix, merge, etc. layers [1]

> Not with weights. I need it to learn case histories to extend its "feature-set".

Again, yes. You can add attention heads, other features, heck you can even add classification if you want [2]. Because you are working with an open architecture! What you think of weights are not binary blobs. That is a common missconception.

[1] - https://github.com/arcee-ai/mergekit

[2] - https://github.com/center-for-humans-and-machines/transforme...

[aloknnikhil]

At first glance, that just seems like a bunch of libraries linked together to form a binary. That is not open-source. I completely agree with you that there is just not enough clarity out there. For my education, following up with my earlier example, can I remove the layers that have references to all chapters / laws in the constitution except for the ones meant for real-estate? How would I do that with the approaches you mentioned here?

Fundamentally, if I have to "reverse-engineer" something, then it's not open-source.

[ErikBjare]

You would have to do the same fine-tuning as if you had the training data.