[1jreuben1]

From a technical perspective, I do not enjoy RAG prompt engineering. It is so brittle and non-formal, and the barriers of entry are extremely low.

There is a complete trifercation of ML: 1. ML Engineers: the high priests, with access to 10K GPU hours, designing novel Transformer architectures using Tensorflow / PyTorch / JAX. 2. Data Scientists: conducting SFT on pre-trained models via the HuggingFace APIs + MLOps & model optimization (eg via TensorRT). 3. GenAI devs: building LangChain orchestrations and RAG prompt flows using off the shelf LLMs commoditized behind APIs - no stats or linear algebra required.

Too many are jumping on this GenAI bandwagon, which will result in a massive hype-cycle trough of dissalusionment and potential VC AI winter.

Furthermore, GenAI is a local maxima on the path to true AGI. COT / REACT heuristics lack the integrated differential aproach of Hybrid AI, ignoring everything previous generations of researchers focused on: problem solving, planning, probabilistic logic, reasoning etc. For true AGI, we need some focus on: 1. concept representation. 2. goal formation. 3. code introspection and self-modification. GenAI is a big distraction from that kind of R&D.

[akaTrickster]

I concur, after LeCun gave the warning to PhD students to steer clear of LLMs I heeded it and can say my career is better as a result: why compete/join a hype cycle if you can help solve foundational problems.

[OutOfHere]

The difference between AGI and LLMs mainly is in the architecture, specifically that LLMs freeze the entire network, whereas AGI would leave some of it perpetually unfrozen so it can update as it goes.