[rusakov-field]

Do you think it will reach "understanding of semantics", true cognition, within our lifetimes ? Or performance indistinguishable from that even if not truly that.

Not sure. I am not so optimistic. People got intoxicated with nuclear powered cars , flying cars , bases on the moon ,etc all that technological euphoria from the 50's and 60's that never panned out. This might be like that.

I think we definitely stumbled on something akin to the circuitry in the brain responsible for building language or similar to it. We are still a long way to go until artificial cognition.

[PlasmaPower]

Why do you think it doesn't have understanding of semantics? I think that was one of the first things to fall to LLMs, as even early models interpreted the word "crashed" differently in "I crashed my car" and "I crashed my computer", and were able to easily conquer the Winograd schema challenge.

[alpaca128]

> even early models interpreted the word "crashed" differently in "I crashed my car" and "I crashed my computer"

That has nothing to do with semantical understanding beyond word co-occurrence.

Those two phrases consistently appear in two completely different contexts with different meaning. That's how text embeddings can be created in an unsupervised way in the first place.

[esafak]

What do you mean? Semantics are determined by distribution. https://en.wikipedia.org/wiki/Distributional_semantics