[blagie]

One related property of GPT-3: It's very bad at traditional computational tasks.

* "Make a list of 20 items" results in a list. The number of items is as accurate as if you asked a toddler the same question.

* If you ask GPT-3 a simple combinatorics question, it will be 100% confident in the wrong answer.

Origami is sort of the same. It takes a conceptual understanding of how paper folds, which DALL-E Mini doesn't have. It has a feel for the general origaminess of a picture.

If I showed a human being a few pieces of origami, including a paper crane, and they had never seen origami before, they'd likely result in similar pictures.

[anon_123g987]

Don't overestimate humans. Most people (adults, not toddlers) can't even draw a bicycle, even if they used one for most of their lives, so presumably they have a conceptual understanding of how it looks and works.

https://www.fastcompany.com/3059089/it-turns-out-its-almost-...

[ossopite]

This example gets trotted out a lot but I don't really understand it. Why do we assume cyclists have a conceptual mechanical understanding of a bicycle and can remember its exact appearance? If they build or repair bicycles, sure, but the majority of people don't do that. They just have learnt how to operate one by instinct.

[riffraff]

I think that's the point, the bike example shows people have some concept of what a bike is (pedals, wheels, frame) even without an exact understanding of it, or a precise image in mind.

It's the same for "draw a house". You may do a square facade, triangular roof, windows, chimney, door. It is equally unrealistic, because people don't have a clue how tall floors should be, have no sense of proportion etc..

It's just that in the bike case it's more obviously wrong.

[blagie]

I feel like we underestimate how close these systems are to sentience, largely by virtue of overestimating humans.

The largest ML systems of today have roughly the same complexity as human brains, and evolve in much the same way. The brain has 100 billion neurons, and GPT-3 has 175 billion parameters. Neurons and parameters aren't comparable, but there isn't an obvious advantage in either direction. Neurons have more parameters than ML parameters, but also operate at around 10Hz, versus many, many MHz.

That doesn't mean machine sentience will be anything like human sentience. Brain disorders are helpful to look at here -- there are people who don't experience specific emotions (e.g. pain, fear, etc.). Even a minor tweak can have a major impact. That's far less than, for example, evolving without evolutionary pressure for self-preservation, for pro-social behavior, or with the sort of ephemeral nature of ML systems.

[aqfamnzc]

Actual webpage for the artist's project: https://www.gianlucagimini.it/portfolio-item/velocipedia/