[nekusar]

Makes sense.

LLMs only have 'knowledge' that was encoded by scanning as much data as possible. (Same way a search engine only can find data that was indexed)

American LLMs are by and large closed. Chinese are open to a point. And research how these things work is still a big mystery.

So yeah, was comprehensive data about saving tokens scanned and indexed? Likely no. So engaging about token saving is going to generate a lot of verbose and useless slop.

[bkotrys]

Yeah, that was pretty much my experience. The models weren’t lacking knowledge as much as discipline. Without a good workflow, they will most likely spend thousands of tokens exploring dead ends.

[nekusar]

I think 'exploring the dead ends' is *possibly* very fruitful. Various disciplines have a lot of appearing dead ends that someone in another field did solve. And they don't talk with each other.

What I've seen with lots of the breathless 'OMG SCIENCE ADVANCEMENT' articles including the one yesterday, is that the LLMs are quite extraordinary about linking a dozen different fields together, and delivering an answer combined from all of them. Basically, the solutions to a lot of current scientific problems are partially solved, but partial from a lot of fields that don't talk with each other. And also, they don't use the same nomenclature, so simple searches don't suffice.

An LLM seems to use Chomsky's Universal Grammar, and thus on training, does normalize all training data, including different words for the same thing.

A lot (perhaps all?) Of these BIG scientific advancements were solved by humans, partially. An LLM jigsawed them together and appeared it solved the thing.

[bkotrys]

Agreed. My goal is not fewer dead ends, but fewer repeated dead ends.

[BobbyTables2]

Sounds like an energetic developer that just graduated from college…

(Without the learning and growth potential)

[jscottbee]

What most AI model users don't understand is (most here will know) that they are probabilistic (statistically based) and not deterministic (static/fixed). So answers to the same prompts can vary (wildly sometimes)

[skeledrew]

This really hit home yesterday as I made a general conversation state logging prompt (because suddenly Claude - Sonnet - was acting far more dumber than expected after /compact), and in 3 different projects it led to Claude saving the logs in 3 different locations (memory, .claude/session_state/.md, docs/.md). Somehow still didn't help increase the smart much BTW, so I'll probably have to engineer it, and just toss /compact out the window.

[nekusar]

Honestly, that sounds more like you triggered the hidden "guard rails" or realignment (aka corporate censorship), and you got downgraded.

Of course, you have no way to he absolutely sure, cause Anthropic lies about silent downgrades, and charges you as if they didnt. And the 'realignment' rules are kept hidden, so you can't even avoid them intentionally.

[skeledrew]

Yeah it's very possible I hit a rail, as one of the projects involves fully automating Claude Code to allow co-driving (similar to what mobile remote control offers, but other agents (eg. Pi) can also be in control, and vice versa). And yes it does seem as though the token burn is also higher, even though it eventually outputs something that works). I'm a patient person though, as long as the rails aren't too restrictive and the subscription value remains reasonable.

[nekusar]

Id recommend getting and running Qwen-3.5-35B-abliterated locally.

Having a local LLM is immensely helpful especially if you do the thing the companies disapprove of. Abliteration guarantees it wont refuse or censor.

The token dealers are incentivised in burning as much tokens as possible, and scamming by crippling/downgrading models. Their goal is to get you to pay more for each pull of the slot machine. And this slot (slop) machine doesn't have state/federal regulations for fairness.

[skeledrew]

Oh yes that reminds me, I have a few small local models including Qwen3.5. The tps rate is pretty bad for interactive use but I should definitely try it for my more automated workflows. I already use DeepSeek (via OR) alongside though and I doubt any Chinese provider will be restrictive in that way, so let Anthropic do what they may :).