Definitely possible on even older off-the-shelf hardware. I use 24GB 4090s for 13b-sized models and have even used 12GB Titans for 7b models, admittedly at much slower rates.
I have a 3080Ti with 12Gb VRAM and would like to try fine tuning the same Mistral 7B model (which I found incredibly potent). Any tips on how to get started?