Warning, link is NSFW, though all it is is a screenshot of my terminal showing the prompt and the output when I loaded mistral-7b and asked it to write an erotic story.
Yes there is indeed a reason to fine-tune. The linked lora will make the model even less likely to refuse than otherwise. Mistral might be willing to do erotica sometimes, but it's often still trying to be "ethical" and not do "immoral" things.
* https://imgur.com/jHlcd2D.jpg
Warning, link is NSFW, though all it is is a screenshot of my terminal showing the prompt and the output when I loaded mistral-7b and asked it to write an erotic story.