[MichaelRazum]

Exactly. It just shows that we can't really control such complex systems. Kind of funny that he got it somehow right. Years ago, I though, nah that can't happen and sounds stupid.

What makes me think that LLM may be a big thing, is that complex language seems to distinguish us from animals. So maybe this is what is required to invent everything else. Or, let's say, at least it is a major factor.

[fullstackchris]

It's really just math though. Any "LLM" isn't really "thinking" in any spoken or written language, but rather in a massive series of weighted matrices (numbers).

I've commented a few times here and there about this AI hype, but might as well repeat myself: I think people largely misunderstand the technology and I see major missing aspects that are non-trivial to solve before we really get to anything looking like iRobot (or insert here any other scifi of your choice). These input / output models can only go so far, even if they are ever increasing in size. We don't just need 2 or 3 prompt memory, but full dynamic memory that the model can access throughout it's lifetime as well as the ability for the model to reflect and introspect on itself (much like human thought and communication). Without these things, an LLM will just remain an LLM, albiet larger and larger. Unfortunately I don't think size for sizes sake will bring much more improvement to such models.

Aside from any of the aforementioned breakthroughs being incorporated, I see this type of chat GPT stuff plateauing in ~1-2 years.

[uejfiweun]

Maybe that's what thinking is though. I mean our brains have neurons that connect to form natural matrices... who's to say that the nature of forcing energy through that mathematical structure isn't the very definition of thinking?

[imtringued]

> Maybe that's what thinking is though. I mean our brains have neurons that connect to form natural matrices... who's to say that the nature of forcing energy through that mathematical structure isn't the very definition of thinking?

I don't know what to say. I think the AI hype just go too far and now people believe random bullshit like in your comment.

Artificial neural networks look nothing like the neurons in our brain. They have very little in common. Artificial neural networks contain layers of neurons where each neuron is connected to all neurons of the previous layer with a floating point weight for each neuron - neuron pair and a floating point bias. This in theory allows you to approximate how the neurons in your brain work but even then it is just an approximation and you may need multiple neurons to simulate a single human neuron.

The next step up is spiking neural networks, which are actually biologically inspired and basically nobody cares about them because back propagation is hard. Why? Because spiking neural networks are not continuous functions. Instead, neurons send spikes and encode information in the timing of their spikes. Neurons only send their own spikes once they cross a certain threshold. So now you have non linear behavior. Again, you can simulate them using ANNs but the primary difference is that spiking neural networks are naturally sparse which is in complete opposition to your statement of "I mean our brains have neurons that connect to form natural matrices". It couldn't be further from the truth. You are now working backwards from the mathematical model of ANNs and are now telling people based on this information how the brain works, despite massive amounts of counter evidence. Do you understand how ridiculous that is? That is only something economists do, because there is money to be made from lying, not biologists or any other science.

[uejfiweun]

Christ almighty, will you calm down, dude? I appreciate the cool info about types of neural nets but god damn, it was just a simple shower thought...

[Moissanite]

> So maybe this is what is required to invent everything else.

A really interesting point. I've always held that we are nowhere close to real AI because we fundamentally don't understand what intelligence is, and we are not building complex enough devices for intelligence to be an emergent property. However, that doesn't consider the possibility that with enough computing power and sufficiently sophisticated models, we could end up with intelligence accidentally bootstrapping itself out of other large models, even if all we are doing is creating linkages between models via API calls and other similarly "dumb" steps.

[inawarminister]

>even if all we are doing is creating linkages between models via API calls and other similarly "dumb" steps.

I mean isn't this what Neuroscience has discovered about the human brain? Millions of year old fish, lizard, and mammals brains... And the neocortex, which is new.

And destroying one part basically damages the whole person.