Hacker News new | ask | show | jobs
by EGreg 412 days ago
Can someone explain how to use Prompt Caching with LLAMA 4?
1 comments
concats 412 days ago
Depends on what front end you use. But for text-generation-webui for example, Prompt Caching is simply a checkbox under the Model tab you can select before you click "load model".
link
EGreg 412 days ago
I basically want to interface with llama.cpp via an API from Node.js

What are some of the best coding models that run locally today? Do they have prompt caching support?

link