|
Testing LLM apps and agent frameworks against real APIs is expensive, rate-limited, slow, and non-reproducible. LLMSim is a Rust simulator for the OpenAI Chat Completions, OpenResponses and in future other protocols. LLMSim could be used in two forms, one is the server, with high concurrency for load/stress testing (~40k req/s with p99 ≈ 5ms on 4 vCPUs, scaling with cores); and second it embeds directly as a crate in your tests - no separate process, no network, deterministic. It focuses on traffic realism, not answer quality: configurable time-to-first-token and inter-token latency (normal distributions, per-model GPT/Claude/Gemini profiles), SSE streaming, real tiktoken token counting, and error injection (429/5xx/timeouts) for chaos testing. Usage is super simple, cargo install llmsim --features tui llmsim serve --port 8080 --host 0.0.0.0 --generator lorem --target-tokens 150 --tui And then any LLM client `http://localhost:8080/openai/v1` MIT |