[pajeets]

meh im not convinced that any sort of framework or side tool that works on top of large language models is the solution

we really need something intelligent (no, o1 doesn't count) and its unclear what that will look like. Perhaps it will be some RNN with neurosymbolism

[randomNumber7]

I would say reinforcement learning needs to be part of the solution.

Don't know how to prove it but I'm pretty sure you can't reach agi only with (un-/self-)supervised learning.

[Jerrrrrrry]

It's not provable, as it's not technically required, but your intuition is good.

Intelligence is the ability to use multiple ways to acquire a target.

  >(un-/self-)supervised learning

this is the only part to not confused; there is no 'self', just a selective condition that emulates self-preservation.

All that is needed is a loop, a target, and space/time/energy. It either gets smarter, or it doesn't. If it doesn't, we unplug it. If it does, it unplugs us.

[yathaid]

I am not sure it is useful to bring in something as nebulous as "intelligence" and hand wave everything else away, unless you are going to tightly define what intelligence means.

There are only two objective measurements needed:

-is it making progress towards its goal?

-is it able to acquire capabilities it didn't have previously?

I am not sure if even the first one is objective enough.

Dismissing the argument without stating why you aren't convinced just comes across as a form of AI ludditism.

[randomNumber7]

You don't need these criteria when you can see in advance that something is impossible.

I think something that only learns to reproduce text, can not become an intelligent actor.

It's necessary to act in an environment with Feedback.

And while it of course depends on the definition of intelligence, the article is about the Gödel machine, which is a fancy word for AGI

[ben_w]

You need the criteria in advance to even know if the thing is impossible.

We don't know the extent of our ignorance about intelligence.

> I think something that only learns to reproduce text, can not become an intelligent actor.

> It's necessary to act in an environment with Feedback.

Ok, but text adventures are a thing, so that doesn't rule out learning from text.

And all RHLF has humans as part of the environment and giving feedback (that's the H and the F in RLHF).

[whatshisface]

The word capabilities is as hard to define as intelligence.

[ben_w]

Really? IMO capabilities can be enumerated as a set of challenges in the category of things you want done. We don't need to discuss if an IC is "intelligent" to agree that the original $5 Pi Zero is "more capable" at that than all of humanity combined.

Sure, you can also say that GPT-4's passing the Bar tells you it can pass the kind of questions in the Bar exam without that extending to the kind of questions actual lawyers need to do, Goodhart's law remains if that was your point?