Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TRT-LLM, and TGI

Y	Hacker News new \| ask \| show \| jobs

	Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TRT-LLM, and TGI (bentoml.com)
	12 points by sherlockxu 745 days ago

2 comments

Hello! We're the authors of this blog post. Please let us know if there are other models and inference backends you'd like us to benchmark next.

Great work