|
|
|
|
|
|
by N_Lens
6 days ago
|
|
|
Interesting repo, but I don't see comprehensive support for MLX/metal backends. MLX performs significantly better on mac hardware - a 100line PR could add this support. Additionally this tool might have a lot of value as a repository for performance optimization data. Getting the right combination of settings, local model, backend, on a particular hardware setup is a bit of a crapshoot. Having a database of benchmarked data with settings/hardware combinations would really help the local llm scene take off. |
|
|