[pembrook]

Copying isn’t just how design works, it’s how everything works. Humans are imitation machines.

We create new things by collecting, regurgitating and mutating stuff we experience, just like LLMs. In a vacuum man has no ideas outside of base impulses.

Hence why originality is a novice belief. The closer you get to any field, the more you realize the stories around who made all the breakthroughs are BS media narratives. Most if not all steps forward in any field have hundreds of people clawing at similar ideas concurrently.

[kivle]

It's very Hackernews to throw LLMs in there, but I agree. LLMs don't have experience though. They have training data, and a probabilistic output.

Designing things have two goals:

- Make old things seem new

- Make new things seem old and familiar

Both need a lot of knowledge about how humans work and how we have made sense of the world up until now. Design can't be made in a vacuum and without input.

Edit: To expand: An LLM would never have come up with touch input. It would have regurgitated the existing ideas of using a pen or a mouse to point at things on a screen. To come up with touch input was a huge feat of human engineering that was a combination of design (making touching a obvious for any human, old or young) and engineering (making that interaction actually work).

[visarga]

> LLMs don't have experience though.

They might not have had experience 2 years ago, but in the meantime they assisted 100s of millions of people for many billion tasks. Many of them are experiences you can't find in a book. They contain on-topic feedback and even real world outcomes to LLM ideas. Deployed LLMs create experiences, they get exposed to things outside their training distribution, they search solution space and discover things. Like AlphaZero, I think search and real world interaction are the key ingredients. For AZ the world was a game board with an opponent, but rich enough to discover novel strategies.

[kivle]

This sounds like an ad. What is "assisted 100s of millions of people in many billions of tasks"? Any real world data? If it's generating new random clip art for presentations, sure. If it's making new flavor text based on generic input, sure.

If my question is "what is the circumference of earth", and I run a model with a temperature of 100, will it give me a good result? Will it always give me a good result with a temperature of 0? I don't think so. It's a huge probabilistic model. It is not an oracle. It can be useful for fuzzy tasks for sure, but not for being smart. You might think it's clever because it's generated code for you, but that's probably because you asked it to make something 500 people already made and published on GitHub.

Edit: Just to clarify. Don't want to step on peoples toes. I just feel like we're at the top of a new dotcom/crypto/nft hype boom. Seen it soooooooo many times before since the beginning of the 2000s. Don't go blind on technology. Research what it actually is. An LLM is a "next word weighted dice toss machine".

[Kiro]

> You might think it's clever because it's generated code for you, but that's probably because you asked it to make something 500 people already made and published on GitHub.

An LLM has no problem coming up with novel solutions within my own esoteric physics framework that only exist on my computer, using my patterns and taking all the nuances into account. It's definitely not just spitting out something it has seen before.

[DJBunnies]

Uhhh, physics?

[Kiro]

Yes, that can be used in a game engine for example.

[visarga]

> What is "assisted 100s of millions of people in many billions of tasks"?

Let's assume 180M users for ChatGPT, each user using the model just 5 times in a month. You got the 1B tasks. If one task uses up 1000 tokens - you have the trillion interactive tokens. It's all based on public data and guesstimation.

[kivle]

And to expand on myself. "Experience" means something very specific for humans. It means you have done something for a long time, and you have failed at it. And you have learned from experience. By definition, LLMs don't have any experience at all. They are trained and become a fresh "brain", and then they make the same mistakes, over and over and over, until they either get a new prompt that might correct themselves or are trained from scratch all over again.

[visarga]

What I meant is

1. LLM generates an idea and

2. the user responds positively or negatively or

3. the user tries the idea and comes back to continue the iteration, communicating the outcomes.

For example the LLM generates some code and I run it, and if it fails I copy paste the error.

That is the (state, action, reward) tuple which defines an experience.

[jpc0]

Sounds like the LLM facilitated a human to gain experience, by making mistakes for the human and then correcting those mistakes also likely in an incorrect way. LLMs are effectively very very bad teachers.

The LLM given the same inputs tomorrow is likely to return similar responses. If a human did that they would likely be concidered to have some sort of medical condition...

[rbits]

I don't know if I buy that an AI wouldn't have been able to come up with touchscreen. It knows people touch screens with pens, and it knows people point and touch things that aren't screens. It could put those ideas together, that's how people came up with

[kivle]

It's not an AI. It's a word probability model. Trained on tons of text. It's not smart at all. The only reason why it might have "figured that out" is because it was actually present in scifi texts in the 60s. The other reason would be that you increased the temperature to something like 100, and then you think you see something genius in some hallucinations, among other unreadable text.

[Lammy]

This is also why Peter Thiel et al are so obsessed with René Girard.

> Philosopher René Girard, scholar Robert Hamerton-Kelly, and Thiel co-founded IMITATIO in 2007 to support the “development and discussion of René Girard’s ‘mimetic theory’ of human behavior and culture.” Mimetic theory, the concept that humans are fundamentally imitative, has had a profound effect on Thiel, who calls Girard “the one writer who has influenced me the most.”

https://www.theverge.com/2016/12/21/14025760/peter-thiel-het...

[Affric]

It goes deeper than that. We are copies of our parents made out of self replicating molecules… copying is fundamental to pretty much everything interesting that has ever happened.

[tracerbulletx]

This is why not much changed for 10s of thousands of years until writing was invented, and accelerated when a valid method of iteration (science) was instituted.