Hacker News new | ask | show | jobs
How good is your AI Gateway? (highflame.com)
3 points by sharathr 10 days ago
3 comments

GoModel author here. I prepared self-reproducible benchmarks and published them on my blog. Might be helpful. I don't think latency is the best metric for comparing AI gateways, as most of the latency comes from the providers. Memory usage is a much better metric and GoModel wins here.

https://enterpilot.io/blog/benchmarking-ai-gateways-gomodel-...

We benchmarked Highflame's AI Gateway against popular alternatives to see how our RUST based unified (LLM/MCP) gateway performs and were surprised at how the competition fared (hint: poorly). We wrote a 2 part blog to review the performance throughput, latency, connection handling in detail.