[imjonse]

Has LoRA been considered as possible alternative for finetuning on your dataset? In that case releasing the 'diff' against the LLaMA weights would be simpler to work with.

[zhisbug]

Yeah that might work but this model wasn’t tuned with lora