GPt-3 is very expensive if you use it frequently compared to just running in a desktop machine you already have. Of course, if you’re buying new hardware just to run a model for yourself locally, that’s a different cost analysis, but for me I had other reasons to have a decent gpu.
If you have a product that uses an LLM and can get away with one of the open source ones, it’s probably cheaper (and def lower latency/response time) to host yourself too somewhere like azure or aws.
If you have a product that uses an LLM and can get away with one of the open source ones, it’s probably cheaper (and def lower latency/response time) to host yourself too somewhere like azure or aws.