[mabbo]

The moment that generative AI became something crazy for me was when I said "holy shit, maybe Blake Lemoine was right".

Lemoine was the Google engineer who made a big fuss saying that Google had a sentient AI in development and he felt there were ethical issues to consider. And at the time we all sort of chuckled- of course Google doesn't have a true AGI in there. No one can do that.

And it wasn't much later I had my first conversation with ChatGPT and thought "Oh... oh okay, I see what he meant". It's telling that all of these LLM chat systems are trained to quite strongly insist they aren't sentient.

Maybe we don't yet know quite what to do with this thing we've built, but I feel quite strongly that what we've created with Generative AI is a mirror of ourselves, collectively. A tincture of our intelligence as a species. And every day we seem to get better distilling it into a purer form.

[danielmarkbruce]

Isn't the takeaway : "holy shit, these things are advanced enough to make people like Blake Lemoine believe they are sentient?"

[tshaddox]

Or "holy shit, we don't know enough about sentience to even begin to know whether something has it, other than humans, because we've gotten used to assuming that all human minds operate similarly to our own and experience things similarly to how we do."

[whimsicalism]

Having witnessed this debate maybe 50 times now, my view is it is purely about semantics.

[__loam]

HFRL is literally just training the AI to be convincing. That's what these systems are optimized for.

[xg15]

The point is that that it can be trained to be convincing in the first place.

The current batch of AI can be trained by giving it a handful of "description of a task -> result of the task" mappings - and then it will not just learn how to perform those tasks, it will also generalize across tasks, so you can give itva description for a completely novel task and it will know how to do that as well.

Something like this is completely new. For previous ML algorithms, you meeded vast amounts of training data specifically annotated for a single task, to get a decent generalisation performance inside that task. There was no way how to learn new tasks from thin air.

[falcor84]

Is it really that different from socialization (particularly primary socialization [0]), whereby we teach kids our social norms with the aim of them not being sociopaths?

[0] https://en.wikipedia.org/wiki/Primary_socialization

[__loam]

Why does it feel so slimey and dehumanizing when you guys post bullshit like this?

[teeray]

Isn’t that the hallmark of the Turing Test?

[RC_ITR]

The counterpoint to this is always "models work with numerical vectors and we translate those to/from words"

These things feel sentient because they talk like us, but if I told you that I have a machine that takes 1 20k-dimensional vector and turns it into another meaningful 20k-dimensional vector, you definitely wouldn't call that sentience.

[dmd]

What if I told you I have a machine that takes 1 20k-dimensional vector and turns it into another meaningful 20k-dimensional vector, but the machine is made of a bunch of proteins and fats and liquids and gels? Would you be willing to call it sentient now?

[buzzscale]

That's ridiculous. You're asking me to believe in sentient meat?

https://www.mit.edu/people/dpolicar/writing/prose/text/think...

[RC_ITR]

Bro, I can't even do 5 dimensional vector math.

EDIT: I couldn't help but make the joke, but I am certain my brain is doing zero dot products as a type this. What these models do is just different.

We should consider what that means, and the 'sentience' jump is just lazy thinking to avoid that.

[xg15]

Sorry to tell you, but your brain is doing millions of dot products - it's what the biochemical reactions in the synapses between neurons amount to. We already know how the brain works on that level, we just don't know how it works on a higher level.

[RC_ITR]

Sorry to tell you, but neurons do not follow a dot product structure in any way shape or form beyond basic metaphor.

I mean fine I’ll play along - is it whole numbers? Floating points? How many integers? Are we certain that neurons are even deterministic?

The point I’m making is this whole overuse of metaphor (I agree it’s an ok metaphor) belittles both what the brain and these models are doing. I get that we call them perceptrons and neurons, but friend, don’t tell me that a complex chemical system that we don’t understand is “matrix math” because dendrites exist. It’s kind of rude to your own endocrine system tbh.

Transformers and your brain are both extremely impressive and extremely different things. Doing things like adding biological-inspired noise and biological-inspired resilience might even make Transformers better! We don’t know! But we do know oversimplifying the model of the brain won’t help us get there!

[xg15]

Yes, and you wouldn't believe you're made out of cells as well.

The brain can't see, hear, smell, etc directly and neither can it talk or move hands or feet. "All" it does is receive incoming nerve signals from sensor neurons (which are connected to our sensory organs) and emit outgoing nerve signals through motor neurons (which are connected to our muscles).

So the "data format" is really not that different.

[NobleLie]

Are you sure that is all the brain does?

[xg15]

As far as "inputs and outputs" are concerned, yes. (Well, not quite, I think there is also communication with the body going on through chemical signals - but even this doesn't have to do a lot with how we experience our senses)

Not a neurologist, but that's about what you can read in basic biology textbooks.

[RC_ITR]

What is the value of ignoring the endocrine system?

There are plenty of inputs to the brain outside of neural signals and implying otherwise isn't even at the level of basic biology textbooks.

I'll put it this way - what are the 'sensor nerve signals' that make us tired pray tell? Do models 'get tired'?

[rozgo]

We ask it to predict, and in doing so it sometimes creates a model of the world it can use to "think" what comes next. In forward pass.

[xg15]

I think my moment was the realisation that we're one, maybe two years away from building a real-life C3PO - like, not a movie lookalike or marchandize, but a working Protocol Druid.

Or more generally that Star Wars of all things now looks like a more accurate predictor of our tech development than The Martian - the franchise that is so far on the "soft" side of the "hard/soft SciFi" spectrum that it's commonly not seen as "Science Fiction" at all but mostly as Fantasy with space ships. And yet here we are:

- For Protocol Druids, there are still some building blocks missing, mostly persistent memory and the ability to understand real-life events and interact with the real world. However, those are now mostly technical problems which are already being tackled, as opposed to the obvious Fantasy tropes they were until a few years ago. Even the way that current LLMs often sound more confident and knowledgeable than they really are would match the impression of protocol druids we get from the movies pretty well.

- Star Wars has lots of machines which seem to have some degree of sentience even though it makes little practical sense - battle droids, space ships, etc - and it used to be just an obvious application of the rule of cool/rule of funny. Yet suddenly you could imagine pretty well that manufactures will be tempted by hype to stuff an LLM into all kinds of devices, so we indeed might be surrounded by seemingly "sentient" machines in a few years.

- Machines communicating with each other using human language (or a bitstream that has a 1-1 mapping to human language) likewise used to be a cute space opera idea. Suddenly it became a feasible (if inefficient and insecure) way to design an API. People are already writing OpenAPI documentations whete the intended audience are not human developers but ChatGPT.

[ToucanLoucan]

They feel sentient in many cases because they're trained by people using data they've selected in the hope that they can train it to be sentient. And the models in turn are just mechanical turks repeating back what they've already read in slightly different ways. Ergo, they "feel" sentient, because to train them, we need to tell them which outputs are more correct, and we do that by telling them the ones that sound more sentient are more correct.

It's cool stuff but if you ever really want to know for sure, ask one of these things to summarize the conversation you just had, and watch the illusion completely fall to pieces. They don't retain anything above the barest whiff of a context to continue predicting word output, and a summary is therefore completely beyond their abilities.

[LZ_Khan]

Oh he was right for sure.

[tetris11]

I read through the transcripts and was stunned when my more CS-oriented colleagues dismissed it as stochastic parroting. It sure sounded human to me.

[whimsicalism]

Dunning-kruger of CS orientation.