Hacker News new | ask | show | jobs
by jacobgold 11 days ago
That's a fair point. I agree that the behavior can look similar even when the mechanism is different.

But in practice, all the analogies I've seen are in fact superficial, including this one.

The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.
3 comments
Kim_Bruning 11 days ago
It's called "code-switching" or "code-mixing", and bilinguals do it all the time. When an immigrant kid does it, you don't call a doctor, you call it adorable.

By the way, it's not switching topic. You just pick the concept closest to what you mean from your combined vocabulary. If you're not paying close attention, you might switch language though (until the next concept you need is from the other language again, at which point you switch back)

And you're aware the paper "Attention is all you need" came out of machine translation research at Google, right? You hold an internal semantic representation and map in and out from arbitrary natural languages. I think the (bi-, tri-, multi-)lingual approach is the only proper way to translate, and this is a hill I will fight on!

Google may have gotten more than they bargained for on that particular translation experiment; though they failed to capitalize on it initially, with OpenAI running with the ball.

link
vidarh 11 days ago
Most of the time when I see an LLM abruptly switching languages it usually continues with something directly related to whatever triggered it.

I'm sure there are other failure modes where it may change topics too, just like humans also regularly digress when triggered by certain words etc.

link
TeMPOraL 11 days ago
What makes you think the similarity is the superficial part, and not the difference in mechanism? I'd argue it's the latter.

> The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.

Haven't met many kids, I see. Or even normie adults talking. I know plenty that tend to jump from topic to topic once they get into a stride talking, and they're not the ones diagnosed with ADHD.

link