[inf3cti0n95]

The problem is, I have to build a generic deployment system for the researchers. To deploy their Model.

Inference frequency would be quite low, but multiple models could be deployed at time.

[pplonski86]

I think generic deployment system can be very difficult to build. For example, there can be problems with library dependence or data format dependence. I think you should select one 'family' of models, for example narrow down to only python scikit-learn models, and try to build a system for them.