|
|
|
|
|
|
by santiago-pl
56 days ago
|
|
|
I've released a new version of GoModel (0.1.20) with explicit support for vllm. You can now use it even with a few vLLM instances. Like this: docker run --rm -p 8080:8080 \
-e VLLM_BASE_URL=http://host.docker.internal:18000/v1 \
-e VLLM_BASEMENT_BASE_URL=http://host.docker.internal:18000/v1 \
enterpilot/gomodel:latest
|
|
|