|
|
|
|
Show HN: A minimalist proxy for your local LLM cluster (~1100 lines)
(github.com)
|
|
2 points
by snowy_robolamp
2 days ago
|
|
|
Hi all! I made this small proxy when I built a small GPU cluster at home and wanted to share it with friends while keeping speed limits and token accounting. I also put a lot of effort in making it work as quick as possible, and IMO the result is worth sharing! It is not trying to be LiteLLM: in fact, I wanted to make it opposite, as small as possible, with a small amount of dependencies and without cloud connections by default. Will be happy to hear your feedback :) PS also installable with pip install smol-llm-proxy |
|
|