I am interested in knowing how the arch would run? Is it a library that I need to add in my code to make it work? Or do I need to deploy some sort of service in my infrastructure?
Its a proxy - built on Envoy. I think its fairly clear that this is a separate process. As far as I can tell, you create a config file, boot up archgw, and in the config have it point to endpoints where prompts get forwarded.