Hacker News new | ask | show | jobs
by brunaxLorax 20 hours ago
I didn't know about Tokenless, the approach seems really innovative, if it works it fixes the "Complexity cannot be deduced from the prompt alone" problem. However you still have the other hidden costs: cache, breaking behavior consistency, and unpredictability.

Your point is interesting, you say that task specificity is easier to classify than task complexity, which I agree - I didn't mention it but we had task specificity routing too :). My opinion is that in many cases task specificity calls are easy to distinguish at build time, and therefore you can isolate them and attach the right model/settings beforehand, so you have less need for a smart routing on the fly.