Check out our open-source tool called Label Studio, https://labelstud.io and we have a commercial offering that does exactly this, finetuning the models.
I actually checked out your github repo and saw your framework that enables integration of trained models into your tool for active learning. Looks awesome! https://github.com/heartexlabs/pyheartex
It would be great if your tool integrates popular object detection frameworks such as TF Object Detection API. Would save people a lot of headache installing and configuring in order to begin finetuning