Interesting repo, but I don't see comprehensive support for MLX/metal backends. MLX performs significantly better on mac hardware - a 100line PR could add this support.
Additionally this tool might have a lot of value as a repository for performance optimization data. Getting the right combination of settings, local model, backend, on a particular hardware setup is a bit of a crapshoot. Having a database of benchmarked data with settings/hardware combinations would really help the local llm scene take off.
Additionally this tool might have a lot of value as a repository for performance optimization data. Getting the right combination of settings, local model, backend, on a particular hardware setup is a bit of a crapshoot. Having a database of benchmarked data with settings/hardware combinations would really help the local llm scene take off.