[huangyz0918]

The project is still under developing, we are trying to make Deep Learning models from academia to industry more easily. With a pre-trained model, ModelCI will convert it to different model types (TensorFlow, Pytorch, etc) and profile it (test throughput, latency) on specific serving devices automatically. Welcome to have a try and join our open-source community. I'm looking forward to your feedback!

[guishen90]

Sounds interesting! I'm a bit confused about how can you profile the system's performance automatically? If I understand correct, the profiling process needs a client to mock real requests in production environment, and the client of serving Deep Learning models are various from one to another.

[huangyz0918]

Yes. Actually, for clients of Deep Learning model, the only difference is the I/O type of the testing data, since there are models of image classification, language model... all of the inputs should be converted into tensor type, we will generate the testing data according to the format of the model's input I/O, and we have some clients in the hub for you if you want to profile the models automatically when you register them in the database. And we also have a parent client class for you to implement by yourself.