Hacker News new | ask | show | jobs
by MitziMoto 1 day ago
5.6 class models have a brand new caching architecture. Make sure you were using the prompt cache breakpoints and cache keys correctly - it's much different than < 5.5 and makes a huge difference for many production workloads.