[pplonski86]

Could you give some more details of what are you trying to build?

What kind of algorithm have you used for model building?

How frequently would you like to request the model for predictions?

Do you need scheduled model updates?

[inf3cti0n95]

The problem is, I have to build a generic deployment system for the researchers. To deploy their Model.

Inference frequency would be quite low, but multiple models could be deployed at time.

[pplonski86]

I think generic deployment system can be very difficult to build. For example, there can be problems with library dependence or data format dependence. I think you should select one 'family' of models, for example narrow down to only python scikit-learn models, and try to build a system for them.