[brutusborn]

Isn't that solved with supervised learning?

The way I see it is that internet content is used to bootstrap the models, then supervision is used to train the models without the risk of a feedback loop causing quality loss.

I'm pretty new to ML so I may be missing something.

[didericis]

Supervision requires knowing the results you want.

Most of these ML projects are essentially just creating a best fit line/shape connecting a huge number of points in multiple dimensions and giving you output coordinates on that line/shape based on your input coordinates (as I understand it). The more supervision, the more you’re negating the value, as you’re basically telling it to make a shape more like something you already understand (instead of something new/actually generative, which requires interesting/novel human input)

I’m not an ML expert either, and if one wants to chime in about how this picture I’m painting is wrong or what else is going on that would be welcome. I’m not trying to belittle how impressive progress has been (I have no idea how the parameters are determined and have a huge amount of respect for people able to handle a hyper-dimensional best fit optimization problem). But I don’t see how all the value isn’t inevitably downstream of high quality human generated digital content, which seems likely to decrease rapidly as more automated content floods the internet and lowers incentives for creators.

[brutusborn]

I mixed up my terms. (I think) supervised learning refers to the use of labelled datasets, I meant fine-tuning (like in RLHF). In either case human input is necessary (and the humans definitely need to know what they want).

In terms of generating novel ideas, I think chatGPT has shown this ability [0]. Human effort will be needed to sort the "good" ideas from the "bad", but I don't think this causes the value of the model to be "negated."

If you want to understand gradient descent and have some math background, this [1] article is a good explainer.

[0] https://forum.effectivealtruism.org/posts/63pYakESGrQpfNw25/... [1] https://towardsdatascience.com/gradient-descent-algorithm-a-...

[didericis]

I understand the basics of gradient descent and was a math major, and while applications of that idea are very cool, I don’t think the weird shapes you can make are genuinely generative. I think that generative capability is illusory, it’s “just” smushing existing content together in a weird way. That best fit shape is inherently derivative and not how true creativity and thought which understands context works.

It’s like a weird parrot. But I think a parrot “understands” more because of the shared embedded evolutionary context it has.

That evolutionary history has the key to true intelligence somewhere, but personally I think it’s inevitably hidden/I don’t think we’ll ever understand how intuition and truly non-derivative, non propositional human thought works.

I also don’t think any of what I’m saying negates the value of these models. These models are fantastic autofill generators for a huge swath of different applications and can vastly improve productivity. I’m saying all this in a lot of threads where it comes up because it seems clear there’s going to be too much enthusiastic adoption, which is going to effectively destroy a lot of value of the internet.

The internet is the best tool for finding genuinely creative and novel ideas you were unexposed to that has ever existed. But it is increasingly dominated by derivative unoriginal content that drowns out what I would argue it was designed to help you find. I have no problem with derivative unoriginal content when it’s properly understood as such. I have a problem with how good these things seem to be at tricking people into following something derivative and blind, which seems very very dangerous.