| Sure! So our model server usually replaces a bunch of docker containers with different standards with zero ways of inputting data. An interesting example was a customer that bought a startup. The startup had a docker container that leaked memory (they were using an advanced TF library, google quit maintaining it). The parent company had tomcat infrastructure. They told the startup to get it to work in tomcat. They approached us. Dl4j itself is built on top of a more fundamental building block that allows us to do some cool stuff in c: https://github.com/bytedeco/javacpp-presets We maintain the above as well. We have our own tensorflow, caffe, opencv,onnx etc bindings. This gives you the same kind of pointer/zero copy interop you would in python but in java. Another area we play in is RPA. You have windows consultants who don't know machine learning but deal with business process data all day. They want standardized APIs and to run on prem on windows:
https://www.uipath.com/webinars/rpa-innovation-week/custom-b... They want the ability to automatically maintain models after they are implemented with an integration that auto corrects the model. We do that using our platform's apis.
On line retraining is another thing we do. We also do automatically rollback based on feedback if you specify a test set. If we find the test is less accurate for a specified metric over time, we roll back the model.
This prevents a problem in machine learning called concept drift which means the domain changes over time. Lastly, you have customers generating their own models and you need to track/audit them all. Sometimes to do online retraining like the RPA use case. Some customers scale to millions of models. They need this kind of tracking. |