[edwin]

With LLMs, the inputs are highly variable so exact match caching is generally less useful. Semantic caching groups similar inputs and returns relevant results accordingly. So {"dish":"spaghetti bolognese"} and {"dish":"spaghetti with meat sauce"} could return the same cached result.

[m3kw9]

Or store as sentence embedding and calculate the vector distance, but creates many edge cases