Check out Substrate - it's an orchestration framework that also runs the inference. https://substrate.run