[danielmarkbruce]

"Designed" is a bit strong. We "literally" couldn't design programs to do the interesting things LLMs can do. So we gave a giant for loop a bunch of data and a bunch of parameterized math functions and just kept updating the parameters until we got something we liked.... even on the architecture (ie, what math functions) people are just trying stuff and seeing if it works.

[JoshTriplett]

https://xkcd.com/1838/

[batshit_beaver]

> We "literally" couldn't design programs to do the interesting things LLMs can do.

That's a bit of an overstatement.

The entire field of ML is aimed at problems where deterministic code would work just fine, but the amount of cases it would need to cover is too large to be practical (note, this has nothing to do with the impossibility of its design) AND there's a sufficient corpus of data that allows plausible enough models to be trained. So we accept the occasionally questionable precision of ML models over the huge time and money costs of engineering these kinds of systems the traditional way. LLMs are no different.

[danielmarkbruce]

Saying ML is a field where deterministic code would work just fine conveniently leaves out the difficult part - writing the actual code.... Which we haven't been able to do for most of the tasks at hand.

What you are saying is fantasy nonsense.

[astrange]

They did not leave it out.

> but the amount of cases it would need to cover is too large to be practical (note, this has nothing to do with the impossibility of its design)

[danielmarkbruce]

It's not only too large - we can't even enumerate all the edge cases, let alone handle them. It's too difficult.

[idiotsecant]

And all you have to do is write an infinite amount of code to cover all possible permutations of reality! No big deal, really.

[jurgenburgen]

Using your logic, we don’t need quantum computers to break encryption, we could just use pen and paper.

[yunnpp]

> would work just fine, but the amount of cases it would need to cover is too large to be practical

So it doesn't work.

[jeremyjh]

It is impossible to design even in a theoretical sense if functional requirements consider matters such as performance and energy consumption. If you have to write petabytes of code you also have to store and execute it.