[ryeights]

The antidote to AI psychosis is not denial. We owe it to ourselves to be humble and consider the possibility that LLM technology leads to general intelligence. The capabilities of coding agents today are certainly outside my wildest imagination circa GPT-3 launch in 2020.

[krupan]

For many of us the release of GPT-3 appeared like a huge step function, a gigantic leap forward, but the fact is, it wasn't that huge. We've been researching computers and AI for around 70 years now. We had productive "machine learning" and pattern completion for a good what, 10 or 15 years before GPT-3 was released. Your phone could auto complete. Computers could win at chess. I worked on working autonomous flying drones back in 2008. In 2020 I joined an autonomous car company that had been slowly grinding for 10 years beforeI joined. The growth is there but not at the blinding rate you think.

One thing I have seen no evidence of is AI being smarter than the smartest humans. It's still just a search engine (with an impressive human language UI) of existing human knowledge. It cannot do anything it has not been trained on.

[ryeights]

In what sense is an LLM analogous (much less reducible to) a search engine?

[krupan]

Search engine: crawl the entire Internet and create an index. When you ask it a question it uses that index to point you to an answer among all the information it crawled.

LLM: crawl the entire Internet and train a model. When you ask it a question, it uses that model to spit out an answer. Worse than a search engine though, if it can't come up with a good answer it just makes shit up

Both use the same sources of information. Neither can produce a (correct) answer that doesn't exist somewhere in that information source

[ryeights]

I think you need a stronger analogy than "they both take questions and produce answers," especially since search engines don't answer questions.