[dotsam]

This article doesn't do a good job of getting at the main points of Valiant's book Educability, in my view. You can see some of them in e.g. this talk he gave here: https://www.youtube.com/watch?v=W4fIoLGjFtM

He makes various arguments in the book that I disagree with, two of which I've put below. On the whole I think it is directionally correct though, and worth reading.

The first quibble I have is about humanity's most characteristic trait. In the book, he writes: "The mark of humanity is that a single individual can acquire the knowledge created by so many other individuals. It is this ability to absorb theories at scale, rather than the ability to contribute to their creation, that I identify as humanity's most characteristic trait".

I don't think that ability to acquire knowledge from other people is our most most characteristic trait. Creativity is. Learning is a form of knowledge-creation, and it is a creative process. We don't passively "absorb" theories when we learn from someone else. Instead, we actively look for and attempt to resolve problems between our existing ideas and the new ideas to create something new.

Another thing I disagree with is when he touches on AGI. He makes the argument that "we should not be fearful of a technological singularity that would make us powerless against AI systems". This is because it will "asymptote, at least qualitatively, to the human capability of educability and no more".

This is reminiscent of David Deutsch's argument that people are universal explainers, and AGI will also be a universal explainer; there is nothing beyond such universality, so they will not fundamentally be different from us (at least, there is nothing that they could do that we couldn't in principle understand ourselves).

I think this is true, but it misses something. It doesn't address the point that there is a meaningful difference between a person thinking at 1x speed (biological human speed) and a person thinking at e.g. 100000x speed (AGI running on fast hardware). You can be outsmarted by something that wants to outsmart you, even if you both possess fully universal educability/creativity, if it can generate orders of magnitude more ideas than you can per unit time. Whether we should be fearful or not about this is unclear, but I do think it is an important consideration.

His overall message though, is good and worth pondering: "Educability implies that humans, whatever our genetic differences at birth, have a unique capability to transcend these differences through the knowledge, skills, and culture we acquire after birth. We are born equal because any differences we have are subject to enormous subsequent changes through individual life experience, education, and effort. This capacity for change, growth, and improvement is the great equalizer. It is possible for billions of people to continuously diverge in skills, beliefs, and knowledge, all becoming self-evidently different from each other. This characteristic of our humanity, which accounts for our civilization, also makes us equal."

[visarga]

> there is nothing that they could do that we couldn't in principle understand ourselves

It is trivial to prove otherwise - AlphaZero move 37. After 4,000 years of gameplay (yes, it is that old!) we still didn't get this level of insight in its strategy.

The core ability of humans might not be learning but search. Creativity is just an aspect of search. AlphaZero was both searching and learning. It's what we do as well, we search and learn. Science advances by (re)search. Art searches for meaningful expression. Even attention is search. Even walking is - where will I place my next foot?

Why is search a better concept that creativity? Because it specifies both the search space and the goal. Creativity, intelligence, understanding and consciousness - all of them - specify only the subjective part, omitting the external, objective part. Search covers both, it is better defined, even scientifically studied.

[HPsquared]

AlphaZero was an example of "learn by doing", not education.

[pixl97]

That's weird, when I went and got an education, a huge portion of that was 'doing'.

[HPsquared]

Fair, education is the wrong word really. That said there's a difference between learning by exercises (AlphaGo) and learning "in production". Synchronous vs asynchronous or something, is there a better name?

[dotsam]

> It is trivial to prove otherwise - AlphaZero move 37. After 4,000 years of gameplay (yes, it is that old!) we still didn't get this level of insight in its strategy.

Are you saying that AlphaZero contains knowledge that we can't understand, even in principle? It is somehow beyond science, beyond all explanation?

> Why is search a better concept that creativity?

Search suggests a fixed set of options, whereas what is crucial is creating new ones.

[fhdsgbbcaA]

A very accomplished older professor once told me in grad school that “research” was a process of first intuiting patterns, and then “searching” for further examples of said pattern, and then “re-searching” until you had statistical confirmation.

I very much agree.

[dotsam]

You don't keep searching until a point of "statistical confirmation". This implies you have arrived at an infallible truth. Instead you look for ways you could be wrong, and try to correct any errors you find.

For instance, if you guess 'all swans are white', you don't ever get "statistical confirmation" that your guess was right. When you eventually see a black swan, you find out you were wrong. Then it's time to come up with a new theory.

[skybrian]

There are very large search spaces. Consider the space of all text documents (Borge's Library of Babel), which includes all research papers and all novels. Also, the space of all mathematical theorems, the space of all images, all videos, all songs, whatever evolution searches over.

These cover many creative activities.

But it's true that some search spaces are less well-defined.

[empath75]

I think one thing we've learned in the past 5 years or so is how inadequate our vocabulary is to describe all the different aspects of intelligence and consciousness (and really just psychology in general). Everything is so handwave-y. What is educability, exactly, in a formal sense -- how could we quantify it and measure it? Mostly we've tried to answer questions like that by writing tests and trying to tease out some reliable measure from it, but that requires so many layers of indirection -- it would be much better to examine the internal state and activity of a "thinking system" directly, something that is rarely possible in humans. I think one way to show that the tests are inadequate is to read the responses to people when those tests are applied to AI. People insist that they simply don't measure what they're supposed to measure in people when they're applied to AI, and for all anyone knows, they may be right -- but _why_? What _exactly_ are those tests measuring, and how could we measure it in a way that _would_ apply to artificial intelligence?

These are philosophical questions that really, despite our best efforts, have never transitioned to a true science, and philosophy has been working on it for thousands of years. We've been hamstrung by the fact that as far as we knew, we were the only intelligent beings in the universe, so it's extremely difficult take any aspect of "the way we think" and separate it, by finding some system that thinks in some ways like us, but in other ways doesn't. It's really only been since we've had large neural networks that anything has approached the way we think in _any_ aspect, so this is probably a once-in-history opportunity to formalize and systematize all of this.

[pixl97]

> it would be much better to examine the internal state and activity of a "thinking system" directly

The particular problem here is complexity management. The 'problem' with thinking is it is a system, very possibly one of the largest and most complex systems we know about. Thinking as we know it in DNA based life is something that's at least 500 million years old and developed a few bits at a time. There is no unwinding the different components of the thinking from each other. Motor skills, reactions, learning, etc are all compressed and mixed together in the same code.

So, going from top down isn't working. But going the other way isn't working either. The computational complexity of a system that gets anywhere close to a biological thinking system requires quadrillions of calculations.

[trott]

> I don't think that ability to acquire knowledge from other people is our most most characteristic trait. Creativity is.

Chimps (and some other animals) can be very creative: https://www.youtube.com/watch?v=fPz6uvIbWZE

[wrs]

“At least qualitatively” is doing a lot of work in that sentence. A computer has the same capability as a human to do any computable algorithm, “at least qualitatively”. But Google search (to name one example) is so far beyond human practical ability that calling it “qualitatively” equivalent is not useful.