| The core pretty much does nothing, except load the plugins and a few functions that we will extract into their components. This is what we did for the other parts. One reason we did this was because we built custom, turn-key, ML products for large enterprise. Complete applications, from data acquisition and model training to "the JavaScript", admin interface, user management, etc. Now... these large enterprise clients were in a sector. We could hardly sell the product to other similar clients because we couldn't just pick and choose which component or features to put on a skeleton. It took us a lot of time, because these projects were both "software engineering" and "machine learning". In other words, we were toast. The worst of both worlds, as we were doing complete applications that even allowed their people to train models themselves. It took a toll on morale. At some point working on eight different projects with different code bases and subsets of the team. We were fed up with this. We wanted to do things differently. We wanted to be able to get the time it took to ship the project the closest possible to the time it took to train models, which we historically did rapidly. It was all the rest that took time. Total time = time to define problem + time to get data + time to produce models + time to write application + a big hairy epsilon We wanted to bring "Total time" to its irreducible form. We didn't want to keep writing different applications for clients. We knew how to do it, but we did it enough times for several clients to notice patterns we wanted to extract into components. We also were losing time with the ML project lifecycle (experiment tracking, model management, collaboration, etc). We didn't want to ask the question "Which model is deployed again? What data produced that model? I tried your notebook on my machine, it doesn't work! DS: Hey, Jugurtha... Can you deploy my model? Jugurtha: I'm busy right now. I'll do it as soon as possible". So we started building our ML platform[0] to remove as much overhead as possible, while being flexible. For example, one of our design goals is that everything one can do on the web app, they should be able to do with an API call. - [0]: https://iko.ai |