[ilaksh]

The real danger in the next few years will be the extreme operating speed of the types of fairly general purpose AIs that we already have such as GPT. This will increase to be dozens of times faster than what humans are capable of. That creates a very strong type of leverage for operators and an incentive to remove slow humans from the loop. Overall transfer of control to these types of systems may lead to a precarious lack of real agency and capability to adapt fast enough for humans. As well as danger from things like smart computer viruses taking over the systems.

I think the only way to be relatively safe from those issues is to limit the hardware performance.

[SanderNL]

I don't want be too human-centric, but to be completely honest we haven't seen the slightest proof that human intelligence is not something special. I know lots of animals are pretty clever, none approach us in any practical sense.

While it looks like an evolutionairy fluke that can be approached or even exceeded by other species - either on this or another planet - in the blink of an eye, I think that's actually more speculative than we would care to admit.

We don't know. Maybe human intelligence is a very close approximation of cognition's equivalent of physic's light speed. Increasing it may turn out to be prohibitively expensive. There's lots of precedence for animals having acquired features close at or actually at the physical maximum of whatever it is they are optimizing for.

To be clear, I'm not convinced of anything either way but I'd think it would as fantastic as it would be slightly depressing to find out human intelligence actually is some kind of global maximum with some exceptions like machines using energy harvested from black hole systems or something.

[ilaksh]

There is a limit to compression of human-relevant information which is largely what intelligence is.

The main thing I am talking about is speed of output. You can already see huge increases in say old GPT-3.5 versus GPT-3.5-turbo or old GPT-4 to new.

We know for a fact that the hardware inference speed can be increased by using faster (currently prohibitively expensive) memory or by packing more onto a chip. There are design for new memory-based computing paradigms.

It's already clear that AI is superintelligent in certain domains or aspects. Such as the ability to exchange information with other agents.

Computer hardware efficiency has relentlessly increased. It would be a total break with history if it suddenly stopped.

[vajrabum]

What do you mean by exchange information with other agents? Can you give a concrete example or two?

[eimrine]

> we haven't seen the slightest proof that human intelligence is not something special.

Have we seen the slightest proof of opposite?

> I know lots of animals are pretty clever, none approach us in any practical sense.

What senses might you consider as enough practical? Have you heard about Koko? What do you think about corvidae?

[SanderNL]

> Have we seen the slightest proof of opposite?

Well, for one, I see no competition. I don't know what the technical definition of "special" is, but I'd say being the only one counts for something.

> What senses might you consider as enough practical? Have you heard about Koko? What do you think about corvidae?

I know both and I know this is a slippery slope. You should know my love for animals runs deep, but I really struggly to put them in the same league as us.

I took a shortcut with saying "practical", because this discussion is way too deep to be performed A) by me and B) on HN. Practical means something like, can they adapt their skills as widely as we can? Can they adapt to uncommon situations? Not subtly or in theory, like solving some puzzle, but really practical? There is nothing subtle about a human becoming a parkour world champignon (I'm leaving this in, just too good) or adapting to life in a submarine (or learning chess, or whittling, or making tea, and many literal millions more examples).

Maybe I am overlooking something, but the skills these animals show seem really minor compared to what even disadvantaged humans are capable of.

[eimrine]

I appreciate the amount of intelligence you put in the message, it is interesting to read and to think about. But the style of your reasoning gives me some hints of creationism, let me show you some anti-creationism point of view.

> Practical means something like, can they adapt their skills as widely as we can?

The most crucial (in my opinion, which has been not introduced to any more crucial points) difference between us and Koko is that we can hold our breath and gorillas can not. That leaded us to develop speech in the seance that speechless group of apes can not win an exactly same group of apes with more developed communicative ability. This, and probably nothing more, has led to such a large gap between humans and apes, so large that humans have ceased to see the relationship between themselves and apes.

I see your understanding of "practical" as something specialized, like agricultural revolution. But why a gorilla should start planting foods if it knows that nobody is going to protect its crops while sleeping because of just lack of common language?

> Can they adapt to uncommon situations?

What can be more uncommon than living on a trees without a warm house and typically without any house at all, without regular nutrition, with a lot of really different enemies from tiny insects to giant cats, with a regular fights, with no democracy and law and medicine?

Being disadvantaged requires to face some uncommonities every day, what about office managers? Disadvantaged people (if they are just poor men and not disabled ones on welfare) can easily survive nuclear war because most of them are OK about living in a similar to gorillas livestyle, but I can not believe that most of average Joes survive a situation when their money are going to cost nothing because of lack of civilization.

[SanderNL]

Oh my, I have seldomly been accused of creationism. To be clear, I can separate the ability from the creature. I don't have a religious or otherwise attachment to the human form specifically. Other than - to be completely honest here - being one.

Let's just clear that out of the way. What I am "claiming", which would be an exaggeration because I'm sort of exploring here, is that whatever human cognition is may be an optimal or near-optimal state of cognitive ability.

So, to be fair, give Koko some millions of years and some evolutionary pressure and I'm sure she'll join us and I'd be happy to have her on our team.

Your point about our ability to hold our breath and how it lead to our increasing dominance is fascinating. I have to say I am not completely sold on the idea that holding your breath is the only way to develop proper channels of communication for I can easily imagine some sort of physical signaling standing in for at least parts of it. That said, I can appreciate the immediate and overwhelming advantage of speech.

This does stimulate my curiosity about what came first here, speech or cognitive ability? Why did "we" even consider speaking? How does one do that without having the cognitive architecture for recognizing its value in the first place? In other words, was "us" being smarter the catalyst for speaking or was it the other way around? Fascinating and I am way too much of an amateur to say anything more of value on it.

I will however continue do so anyway, because that is my sacred duty as a dedicated HN'er and allround developer douchebag.

> What can be more uncommon than living on a trees without a warm house and typically without any house at all, without regular nutrition, with a lot of really different enemies from tiny insects to giant cats, with a regular fights, with no democracy and law and medicine?

I might be in danger of being too blunt here, but this is the bar you have to clear if you wish to survive. This is exactly what humans are capable of even in their "undeveloped" form. These sort of pressures might be foundational to our evolution, but then again, every animal has to deal with it in some way or another so I'm not sure what made us take what I can only call the excessively cerebral path. Maybe it was like the evolution of the peacock's tail? A runaway process, leading to miraculous but exorbitant results like the mantis shrimp's eyes.

What I mean by uncommon is: can we coach you to pick cotton, whittle little wooden sculptures, play a game like checkers and sing simple songs or whatever else is appropiate for your particular physical form and has virtually no bearing on your immediate survival? I know this is a hard thing to pin down, because one can come up with myriad examples of varying levels of persuasive power but you surely perceive some differences here even if they are hard to lock into? Differences that cannot just be attributed to language or lack of proper motivation.

It's not so much every thing we can do in particular that's piquing my interest, but the sheer breadth of things we are capable of taking on both physical (parkour, gymnasts) and cerebral (chess, math). I didn't even get to art, which is like a whole world on its own and the various combinations of all those domains.

[eimrine]

> This does stimulate my curiosity about what came first here, speech or cognitive ability?

This is the question I thought about all evening before I fell asleep. I have two ways to answer it.

1. Let's take the well-known Feline and Canine. All my friends who spend a lot of time with animals will call dogs smarter than cats, but why? Dogs have a more developed communication system: they have more varieties of barking than cats have varieties of meowing. Dogs are playful, they know how to smile, they know how to feel guilty and actively show it, they are capable of paired activities under the supervision of a person. From what most of dogs can't, cats can only chase prey without visual or odor contact, purely by sound (but polar foxes can do even this). Conclusion - the level of communication correlates with the level of intelligence.

2. Let's take the most primitive organism, the prokaryote (sorry for not naming some precise specie, let's consider some abstract prokaryote with the requirement to be the simplest). Google tells us: > All organisms, from the prokaryotes to the most complex eukaryotes can sense and respond to environmental stimuli.

But also Wikipedia tells us that prokaryotes are able to interchange some information using DNA: > These are (1) bacterial virus (bacteriophage)-mediated transduction, (2) plasmid-mediated conjugation, and (3) natural transformation.

These two examples make me confident in the opinion that communication and cognition are two different words for describing the same idea from two different points of view.

[macintux]

I’m not weighing in on the debate, but there does seem to be a great deal of doubt that Koko was using language in an intelligent way.

[eimrine]

Even when she reported about her toothache? If this situation was not fabricated (why to fabricate a work of your life and how to do it unnoticed) the situation meets the definition of intelligence.

What are your doubts, are they related on some data?

[macintux]

https://en.wikipedia.org/wiki/Koko_(gorilla)#Criticism

“Great deal” may be overstated.