[derrak]

Maybe you’re right about modern LLMs. But you seem to be making an unstated assumption: “there is something special about humans that allow them to create new things and computers don’t have this thing.”

Maybe you can’t teach current LLM backed systems new tricks. But do we have reason to believe that no AI system can synthesize novel technologies. What reason do you have to believe humans are special in this regard?

[adamiscool8]

After thousands of years of research we still don’t fully understand how humans do it, so what reason (besides a sort of naked techno-optimism) is there to believe we will ever be able to replicate the behavior in machines?

[coldtea]

Well, understanding how it works is not a prerequisite to being able to do it.

People have been doing thigs millenia before they understood them. Did primitive people understood the mechanism behind which certain medicinal plants worked in the body, or just saw that when they e.g. boil them and consume them they have a certain effect?

[stravant]

Thousands of years?

We've only had the tech to be able to research this in some technical depth for a few decades (both scale of computation and genetics / imaging techniques).

[thesz]

And then we discover that DNA in (not only brain) cells are ideal quantum computers, DNA's reactions generate coherent light (as in lasers) used to communicate between cells and single dendrite of cerebral cortex' neuron can compute at the very least a XOR function which requires at least 9 coefficients and one hidden layer. Neurons have from one-two to dozens of thousands of dendrites.

Even skin cells exchange information in neuron-like manner, including using light, albeit thousands times slower.

This switches complexity of human brain to "86 billions quantum computers operating thousands of small neural networks, exchanging information by lasers-based optical channels."

[derrak]

The Church-Turing thesis comes to mind. It would at least suggest that humans aren’t capable of doing anything computationally beyond what can be instantiated in software and hardware.

But sure, instantiating these capabilities in hardware and software are beyond our current abilities. It seems likely that it is possible though, even if we don’t know how to do it yet.

[sophrosyne42]

The church turing thesis is about following well-defined rules. It is not about the system that creates or decides to follow or not follow such rules. Such a system (the human mind) must exist for rules to be followed, yet that system must be outside mere rule-following since it embodies a function which does not exist in rule-following itself, e.g., the faculty of deciding what rules are to be followed.

[derrak]

We can keep our discussion about church turing here if you want.

I will argue that the following capacities: 1. creating rules and 2. deciding to follow rules (or not) are themselves controlled by rules.

[Verdex]

Church turing is about computable functions. Uncomputable functions exist.

For example how much rain is going to be in the rain gauge after a storm is uncomputable. You can hook up a sensor to perform some action when the rain gets so high. This rain algorithm is outside of anything church turing has to say.

There are many other natural processes that are outside the realm of was is computable. People are bathed in them.

Church turing suggests only what people can do when constrained to a bunch of symbols and squares.

[ben_w]

That example is completely false: how much rain will fall is absolutely a computable function, just a very difficult and expensive function to evaluate with absurdly large boundary conditions.

This is in the same sense that while it is technically correct to describe all physically instantiated computer programs, and by extension all AI, as being in the set of "things which are just Markov chains", it comes with a massive cost that may or may not be physically realisable within this universe.

Rainfall to the exact number of molecules is computable. Just hard. A quantum simulation of every protein folding and every electron energy level of every atom inside every cell of your brain on a classical computer is computable, in the Church-Turing sense, just with an exponential slowdown.

The busy beaver function, however, is actually un-computable.

[Verdex]

The busy beaver function isn't uncomputable.

You just compute the brains of a bunch of immortal mathematics. At which point it's "very difficult and expensive function to evaluate with absurdly large boundary conditions."

[ben_w]

> The busy beaver function isn't uncomputable.

False.

To quote:

  One of the most consequential aspects of the busy beaver game is that, if it were possible to compute the functions Σ(n) and S(n) for all n, then this would resolve all mathematical conjectures which can be encoded in the form "does ⟨this Turing machine⟩ halt".[5] For example, there is a 27-state Turing machine that checks Goldbach's conjecture for each number and halts on a counterexample; if this machine did not halt after running for S(27) steps, then it must run forever, resolving the conjecture.[5][7] Many other problems, including the Riemann hypothesis (744 states) and the consistency of ZF set theory (745 states[8][9]), can be expressed in a similar form, where at most a countably infinite number of cases need to be checked.[5]

"Uncomputable" has a very specific meaning, and the busy beaver function is one of those things, it is not merely "hard".

> You just compute the brains of a bunch of immortal mathematics. At which point it's "very difficult and expensive function to evaluate with absurdly large boundary conditions."

Humans are not magic, humans cannot solve it either, just as they cannot magically solve the halting problem for all inputs.

[ben_w]

That humans come in various degrees of competence at this rather than an, ahem, boolean have/don't have; plus how we can already do a bad approximation of it, in a field whose rapid improvements hint that there is still a lot of low-hanging fruit, is a reason for techno-optimism.

[analog31]

>>> But do we have reason to believe that no AI system can synthesize novel technologies

We don’t even know if they want to. But in general, it’s impossible to conclusively prove that something won’t ever happen in the future.

[parpfish]

Something I think about frequently is that 20 years ago, there weren’t machines that could do visual object recognition/categorization and we didn’t really have a clue how humans did it either. We knew that neuron built fancier and fancier receptive fields that became “feature detectors”, but h the ere was a sense of “is that all it takes? There has to be something more sophisticated in order to handle illumination changes of out of plane rotation?”

But then we got a neural wr that was big enough and it turns out that feedforward receptive fields ARE enough. We don’t know whether this is how our brains do it, but it’s a humbling moment to realize that you just overthought how complex the problem was.

So ive become skeptical when people start claiming that some class of problem is fundamentally too hard for machines.

[slopinthebag]

Are modern visual recognition & categorisation systems comparable to human capabilities? From what I can tell, they aren't even close (although still impressive!).

[LtWorf]

They aren't, or capchas wouldn't be a thing any longer.

[somewhereoutth]

In the grand scale of things, a computer is not much more than a fancy brick. Certainly it is much closer to a brick than to a human. So the question is more 'why should this particularly fancy brick have abilities that so far we have only encountered in humans?'

[derrak]

> Certainly it is much closer to a brick than to a human.

I disagree with this premise. A computer approximates a Turing Machine, which puts it far above a brick.

[somewhereoutth]

but still so so much further to go until you reach human.

[djeastm]

> fancy brick

If we're going to be reductionist we can just call humans "meat sacks" and flip the question around entirely.

[danaris]

That's irrelevant.

The claim being made is not "no computer will ever be able to adapt to and assist us with new technologies as they come out."

The claim being made is "modern LLMs cannot adapt to and assist us with new technologies until there is a large corpus of training data for those technologies."

Today, there exists no AI or similar system that can do what is being described. There is also no credible way forward from what we have to such a system.

Until and unless that changes, either humans are special in this way, or it doesn't matter whether humans are special in this way, depending on how you prefer to look at it.

[derrak]

Note that I prefaced my comment by saying the parent might be right about LLMs.

> That's irrelevant.

My comment was relevant, if a bit tangential.

Edit: I also want to say that our attitude toward machine vs. human intelligence does matter today because we’re going to kneecap ourselves if we incorrectly believe there is something special about humans. It will stop us from closing that gap.

[sophrosyne42]

Its not an assumption, it is a fact about how computers function today. LLMs interpolate, they do not extrapolate. Nobody has shown a method to get them to extrapolate. The insistence to the contrary involves an unstated assumption that technological progress towards human-like intelligence is in principle possible. In reality, we do not know.

[derrak]

As long as agnosticism is the attitude, that’s fine. But we shouldn’t let mythology about human intelligence/computational capacity stop us from making progress toward that end.

> unstated assumption that technological progress towards human-like intelligence is in principle possible. In reality, we do not know.

For me this isn’t an assumption, it’s a corollary that follows from the Church-Turing thesis.

[sarchertech]

That certainly doesn’t follow from the Church-Turing thesis because the Church Turing thesis doesn’t demonstrate that human intelligence is computational. That it is still an unstated assumption.

[derrak]

Noted, thanks.

I don’t know what non-computational intelligence would look like but I guess I’ll keep my mind open.