[StevenWaterman]

> model welfare

I shouldn't get in arguments about this stuff online, but have you actually sat down and thought about this in-depth? It's pretty normal to have a gut reaction that this is insane, but it's worth thinking about.

If we replaced every neuron in your brain, one by one, with a silicon equivalent, would you stop being conscious? At what point? Would you be able to tell?

If yes, what about if we took those silicon neurons and instead simulated them, IE perfectly simulated an entire human brain, neuron-for-neuron, would that be conscious?

If yes, where is the line at which code starts being conscious? Can you categorically say that existing LLMs are on one side or the other? Would scaling up or down change that?

There are robust philosophical positions on all sides of this argument.

[georgeecollins]

>> If we replaced every neuron in your brain, one by one, with a silicon equivalent, would you stop being conscious?

If we put your brain in a jar, would you be a person? Would you have normal consciousness? Honestly I don't know and I am not the person to ask.

But the brain is just a computer metaphor has flaws, particularly when you take account how poorly the brain is understood.

[StevenWaterman]

Yeah absolutely, I have no objections to you looking at my thought experiments and saying "No, if you replaced every neuron with silicon I would stop being conscious". That's a totally valid conclusion and is a mainstream philosophical position. That's why they're thought experiments.

My issue was more just that people see the concept of a thinking machine and immediately reject it because it sounds silly, without thinking about it.

[My_Name]

It has been claimed in the past that computers would never think "because they are made of matter"

It was a serious statement at the time it was made, but it is clearly ludicrous. Like the universe created special states for arranging energy in just to make the human brain from.

[My_Name]

>>If we put your brain in a jar, would you be a person?

This is very close to the question : Would you still be a person if your arms and legs were amputated, and you could not communicate.

I would hope you would see that the answer is : Of course you would be

[Lutzb]

ARTHUR I'm sorry -- did you just say you need my brain?

BENJY MOUSE Yes. To extract the question.

ARTHUR You can't have my brain. I'm using it.

FRANKY MOUSE Barely.

BENJY MOUSE We can replace it if you think it's important. An electronic brain maybe.

FRANKY MOUSE A simple one should suffice. Who would know the difference?

ARTHUR I would!

FRANKY MOUSE No, you wouldn't. We could program you not to.

[metadat]

Source? This is gold.

[commoner]

The Hitchhiker's Guide to the Galaxy

- Script: https://archive.org/stream/hitchhikers-guide-to-the-galaxy/H...

- Film (with some differences): https://www.youtube.com/watch?v=41Vli6Fnj1E

[AlexeyBrin]

Probably Douglas Adams The Hitchhiker’s Guide to the Galaxy the book or the screenplay for the movie with the same name.

[Lutzb]

It's from the book.

[Lutzb]

Sorry, meant to say it's from the movie script. Need more sleep.

[AlexandrB]

> If we replaced every neuron in your brain, one by one, with a silicon equivalent, would you stop being conscious? At what point? Would you be able to tell?

The idea that we could have a silicon equivalent at all is already a huge assumption. For starters, the silicon equivalent cannot grow or kill neural connections or neurons as needed. Biological systems have a number of properties that silicon systems just don't at the moment.

[ngruhn]

But everything that's different seems like an arbitrary requirement. I hope I'm also conscious in moments where I'm not actively growing/killing neural connections.

[AlexeyBrin]

You can probably simulate (in theory) the creation and destruction of neural connections the same way we simulate "wires" between elements in an FPGA.

[My_Name]

Ah, the brain of Perseus.

If, as you were replacing neurons, you put them together in the same arrangement in the jar next door so you had 2 exact copes, which one is your brain?

At the exact point 50% of your neurons are copied, which one is your brain?

The best question is : does it matter?

These are similar to the contemplations that resulted in 'cogito ergo sum', being that all you can prove is that you have thoughts, so on some level, somewhere there is something having those thoughts : "I think, therefore I am"

[rootlocus]

There's a reason we have expressions like "train of thought". Going from one idea to another is pretty much how we think. Much like going from one token to another.

[AlexandrB]

Not true in 2 particular ways:

1. "Eureka moments" often pop up almost out of nowhere, this suggests that there's processing going on at a non-verbal, non-token level.

2. The fact that we think in language is an evolutionarily new phenomenon. Plenty of "smart" animals are capable of complex behaviors and learning without any kind of language "tokens". There's no reason to think that human reasoning is different and somehow confined to "tokens" instead of being a broader system. Consider language itself. The human species created it somehow out of nothing. We "tokenized" our cognition to be able to communicate. But I would argue this was driven by our social needs, not our cognitive needs and the human ability to think does not require tokens at all.

Edit:

Finally, consider how limited language really is. Even for something simple, like pain, we don't even have a way to communicate objective reality to another person. We can say "it hurts" and use metaphors: "stabbing pain", "dull pain", etc. But the experience of pain can't really be communicated to another person. It's even worse once you remove non-verbal cues like facial expressions and body language - pain just becomes a vague concept on the page.

Meanwhile, our brain is perfectly capable of perceiving the slightest changes in pain and altering our behavior accordingly. Now extend this to other human experience.

I think the best analogy here is that treating human cognition as a problem of processing "tokens" is confusing the map for the territory.

[morbicer]

Then they should stop the development and focus on the identifying of consciousness. But wait, there's money to be made now so screw that and just talk big words to press.

[everforward]

I don’t even have to try to address whether models or conscious or not to point out the absurdity.

8.3% of US households had low food security at some point in 2024 (read: had to resort to less varied diets like rice and beans, or visit a food pantry).

5.4% had very low food security at some point in 2024 (at least one family member had to disrupt their eating patterns and skip meals or eat less due to food availability).

Those numbers don’t overlap, it’s 13.7% experienced some variety of food insecurity.

It’s worse in families with children, 18.4% experienced some form of food insecurity. Thankfully, even in most households with very low food security the adults bear the brunt and make sure the kids eat enough, but in 0.9% of households even the children have very low food security.

Talking about welfare for AI models is utterly insane while those numbers are above 0%. I do not give a single shit about model welfare when we have human children skipping meals. Frankly it’s the inverse and if Anthropic has money to piss away on this then we should increase their tax rate and feed some kids, fix some roads, do some good. They can go look at model welfare after they’ve chucked us into the post-scarcity universe they keep telling us is 2 model releases away.

[solenoid0937]

There's no point. HN is arrogant and thinks it understands the subject matter better than the people that have worked on it their whole lives.

Everything is a conspiracy here.

[AlexeyBrin]

There is no HN as a thinking entity, there are individual users each with his own mind.

[solenoid0937]

Seems to behave very much like a hive mind in practice. 97% of the comments here share the same misinformed view.