|
|
|
|
|
|
by andai
58 days ago
|
|
|
>IMO, a key development will be having an "overlay" of weights to apply on top of a "clean" world model that is tailored to whatever interests can pay for it. Being able to serve that overlay dynamically, or atleast per-user is the killer app. You mean LoRA? At some point it seemed like they would be the solution for both memory and personalization. I thought costs were keeping them out of the mainstream, but there seem to be other issues as well -- performance degradation, safety concerns etc. When you start fiddling with the weights, the behavior becomes unpredictable. (The fine tuning endpoints appear to be powered by LoRA.) We saw this most dramatically with that paper that found fine tuning GPT to produce code with exploits also made it evil in conversational contexts: https://news.ycombinator.com/item?id=43176553 |
|
|