Hacker News new | ask | show | jobs
by Groxx 16 hours ago
Running some local models and wondering what insanity was consuming 100k+ tokens to respond to "hello world" was rather eye-opening, yeah. Full of pointless fluff.
1 comments

I think prompt engineering went way to far into verbosity when at the same time agent and tool calling are much more important. Now a small system prompt can grab the contect it needs based on user context before bloating. On local models, i use dynamic context pruning.