[PaulHoule]

Certain people always say that "art" isn't "art" but it's a consistent way to embarrass yourself whether the object is Duchamps' "Readymades" or video games (in the case of a famous movie reviewer.)

Particularly any kind of "A.I." is always considered by some to not be "A.I." for instance a chess playing program is just searching moves, an expert system is just applying rules, software that lays out microchips is just solving an optimization problem, etc.

It is moving the goalposts and it is a form of ignorance that leaves the field wide open to the likes of Eliezer Yudkowsky. Particularly, like the aphorism that "an LLM can't create anything new" it distracts people from the serious task of figuring out what specific things these things can and cannot do.

[smoldesu]

> the serious task of figuring out what specific things these things can and cannot do.

Is it rude or reductive to suggest that we already know what they can and cannot do? It's just text. Text can be interpreted in meaningful ways, and it's cool that this text can react to user input, but it's still... starkly limited. I'm honestly not sure what serious things are left to figure out with AI. It's like the Library of Babel in a way, it contains both everything and nothing. But it's also just entropy limited to text, which in-and-of itself is not that powerful.

It's heady stuff and I don't know if there are any right answers. I feel like it's capabilities are not as strong as others profess though.

[PaulHoule]

No, it is an active research area to discover what they can do. A typical paper is something like this

https://arxiv.org/abs/2305.18618

My opinion is as follows:

Frankly the thing they are best at is bullshitting and some people are particularly good at falling for it, many of those people think they are talented "prompt engineers". I think there is something about picking the "most likely" next word that stops people from perceiving incongruities, which gives chatbots a hypnotic power.

There are certain kinds of problems chatbots seem to do well on but usually research shows they struggle to do anything right more than 80% of time. For some problems ("is this about astrophysics or psychology?") you can put the same LLM into a supervised training situation and get the right answer 99% of the time.

Chatbots are a big pile of biases and frequently come to correct answers by following shortcuts. The real life criminal justice system is biased in various ways but the chatbot would make a verdict based on a number of things it read like "Tyrone is a thug", in a conventional situation where the usual assumptions apply it looks genius, but if you go off the usual rails you find they stay on them.

I've been interested in NLP both in terms of side projects and work since 2004 or so and for long I believed the Chomskyian idea that the "language instinct" is a peripheral that bolts onto an animal and that most of your human intelligence is really intelligence you share with mammals and birds at the very least. Computers have struggled to process language because they lack the groundedness in the world that animals have. I'd contrast that to this discredited trend in philosophy

https://en.wikipedia.org/wiki/Structuralism

which thought language had intrinsic meaning and is a model for understanding other things in the social sphere. Chatbots really do accomplish a lot more with text alone than I and many other people thought were possible and will actually revive structuralism. Yet boy do they bullshit and it is scary to see how giddy people get when they are seduced by them and think about how they'll soon be pressed into service running "pig butchering" and other romance scams.

Personally I feel really jealous because they seem to elicit the neurotypical privilege I never had.