[Unearned5161]

I have a pet peeve with the concept of "a genuinely novel discovery or invention", what do you imagine this to be? Can you point me towards a discovery or invention that was "genuinely novel", ever?

I don't think it makes sense conceptually unless you're literally referring to discovering new physical things like elements or something.

Humans are remixers of ideas. That's all we do all the time. Our thoughts and actions are dictated by our environment and memories; everything must necessarily be built up from pre-existing parts.

[davidfarrell]

W Brian Arthur's book "The Nature of Technology" provides a framework for classifying new technology as elemental vs innovative that I find helpful. For example the Huntley-Mcllroy diff operates on the phenomenon that ordered correspondence survives editing. That was an invention (discovery of a natural phenomenon and a means to harness it). Myers diff improves the performance by exploiting the fact that text changes are sparse. That's innovation. A python app using libdiff, that's engineering. And then you might say in terms of "descendants": invention > innovation > engineering. But it's just a perspective.

[0x3f]

Novel things can be incremental. I don't think LLMs can do that either, at least I've never seen one do it.

[A_D_E_P_T]

Suno is transformer-based; in a way it's a heavily modified LLM.

You can't get Suno to do anything that's not in its training data. It is physically incapable of inventing a new musical genre. No matter how detailed the instructions you give it, and even if you cheat and provide it with actual MP3 examples of what you want it to create, it is impossible.

The same goes for LLMs and invention generally, which is why they've made no important scientific discoveries.

You can learn a lot by playing with Suno.

[hodgehog11]

I don't see how this is an architectural problem though. The problem is that music datasets are highly multimodal, and the training process is relying almost entirely on this dataset instead of incorporating basic musical knowledge to allow it to explore a bit further. That's what happens when computer scientists aim to "upset" a field without consulting with experts in said field.

[chpatrick]

https://news.ycombinator.com/item?id=46094037

[bonesss]

Genuinely novel discovery or invention?

Einstein’s theory of relativity springs to mind, which is deeply counter-intuitive and relies on the interaction of forces unknowable to our basic Newtonian senses.

There’s an argument that it’s all turtles (someone told him about universes, he read about gravity, etc), but there are novel maths and novel types of math that arise around and for such theories which would indicate an objective positive expansion of understanding and concept volume.

[chpatrick]

Einstein was heavily inspired by Mach: https://en.wikipedia.org/wiki/Mach%27s_principle

[jungturk]

Nah - Poincare & Lorentz did quite a bit of groundwork on relativity and its implications before Einstein put it all together.