|
|
|
|
|
|
by ixaxaar
818 days ago
|
|
|
Hey I did, and sorry for the self promo, Please check out https://github.com/geniusrise - tool for running llms and other stuff, behaves like docker compose, works with whatever is supported by underlying engines: Huggingface - MPS, cuda
VLLM - cuda, ROCm
llama.cpp, whisper.cpp - cuda, mps, rocm Also coming up integration with spark (TorchDistributor), kafka and airflow. |
|
|